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CHAPTER  1 


INTRODUCTION 

1.1  Motivation 

In  large  organizational  structures  where  the  decision  making  process  is  too  compli¬ 
cated  to  be  handled  by  a  single  agent  (or  decision  maker)  or  where  agents  are  distributed 
over  a  wide  geographic  area,  decentralized  decision  making  paradigm  is  employed  fre¬ 
quently.  Examples  of  such  decision  making  organizations  occur  in  financial  institutions, 
industrial  environments,  and  military  surveillance  systems  etc.  In  order  to  make  the 
group’s  decision  more  efficient  and  to  avoid  the  dominance  of  an  individual  member  over 
the  rest,  the  group  is  divided  into  smaller  decision  making  units  consisting  of  one  or  more 
members  (expens).  Decisions  of  these  smaller  units  are  conveyed  to  the  head  of  the  orga¬ 
nization  who  is  responsible  for  decision  combining  and  reaching  the  final  decision. 

One  interesting  application  of  group  decision  making  that  has  received  an  increasing 
interest  in  recent  years  is  the  design  and  analysis  of  distributed  sensor  networks  for  signal 
detection  (decentralized  detection  systems).  This  is  due  to  the  advantages  that  these  sys¬ 
tems  have  over  their  centralized  counterparts  like  reliability,  survivability,  and  shorter  de¬ 
cision  times.  In  a  decentralized  detection  system  there  is  a  group  of  remotely  located  local 
detectors  that  monitor  a  common  phenomenon.  These  detectors  are  linked  to  a  primary  de¬ 
tector  (also  known  as  the  global  decision  maker  or  data  fusion  center)  through  bandlimited 
channels.  Due  to  this  limitation  on  the  bandwidth,the  local  detectors  are  not  able  to  convey 
to  the  primary  detector  all  the  information  available  to  them  about  the  phenomenon.  In¬ 
stead,  they  provide  the  primary  detector  with  a  compressed  version  of  their  data.  The  role 
of  the  primary  detector  is  to  combine  the  preliminary  decisions  of  the  local  detectors 
along  with  any  locally  received  observation  to  make  the  final  decision.  For  a  given  opti¬ 
mality  criterion,  the  design  of  a  decentralized  detection  system  involves  specifying  both 
the  local  decision  rules  and  the  global  decision  rule.  Due  to  the  constraints  on  the  trans- 
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mission  capacity  of  the  communication  channels,  the  system  experiences  performance 
degradation  and  computational  difficulty  in  the  design  of  the  optimal  system.  In  the  ab¬ 
sence  of  these  constraints,  the  configuration  reduces  to  a  centralized  system  for  which  a 
well  developed  theory  exists  [1-3].  Because  of  the  full  utilization  of  the  raw  observations 
in  the  centralized  system,  its  performance  is  used  as  a  benchmark  for  comparing  the  per¬ 
formance  of  different  decentralized  detection  network  structures  based  on  different  design 
criteria.  In  general,  the  design  of  an  optimum  decentralized  detection  configuration  is  a 
computationally  difficult  task  [4].  Therefore,  rather  than  resorting  to  the  optimum  design 
criterion,  suboptimum  design  criteria  may  be  used  instead.  A  design  criterion  A  is  said  to 
be  “better”  than  a  design  criterion  B  if  the  performance  of  the  system  when  criterion  A  is 
employed  is  closer  to  the  performance  of  the  centralized  system  than  the  performance  of 
the  system  when  criterion  B  is  employed. 

Figures  1.1  -  1.4  show  four  decentralized  detection  configurations  that  are  treated  in 
this  dissertation.  The  parallel  fusion  system  shown  in  Fig.  1.1  represents  the  simplest  type 
of  decentralized  detection  structure.  It  consists  of  n  local  detectors  and  a  fusion  center. 
Due  to  the  simplicity  of  its  structure,  the  design  of  the  system  subject  to  different  design 
criteria  has  received  most  of  the  interest  in  the  literature.  This  system  is  used  for  hypothe¬ 
sis  testing.  In  this  dissertation,  we  restrict  our  attention  to  binary  hypothesis  testing  prob¬ 
lems  where  the  null  hypothesis  Hq  is  tested  against  the  alternative  hypothesis  Hj.  The 
system  receives  n  observations  Xj,  X2,...,  Xn  in  which  Xj  is  the  observation  received  by 
the  local  detector  LD,  (in  the  classical  detection  theory,  it  is  assumed  that  all  these  raw  ob¬ 
servations  are  available  at  one  central  location  and  that  the  decision  is  made  based  on  the 
entire  set  of  observations).  Due  to  the  presence  of  noise  in  a  practical  system,  these  obser¬ 
vations  are  assumed  to  be  continuous  random  variables  with  conditional  density  functions 
Pj(xj) ;  j=0,l  ,  i=0,l,...,n.  From  an  information  theoretic  point  of  view,  the  transmission  of 
the  information  in  these  observations  from  the  local  detectors  to  the  global  decision  maker 
requires  infinite  capacity  channels.  A  requirement  which  is  not  practically  attainable.  To 
overcome  this  problem,  each  local  detector  processes  the  locally  received  observation  Xj 
and  transmits  a  compressed  version  Zj  of  the  data  to  the  global  decision  maker.  The  way  in 
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Phenomenon  H 


Fig.  1.1.  Decentralized  detection  system  with  fusion,  Sj. 
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Phenomenon  H 


which  data  reduction  takes  place  can  be  in  the  form  of  a  hard  decision  or  a  soft  decision.  In 
the  first  case,  the  local  decision  Zj  takes  on  one  of  two  possible  values  indicating  the  pres¬ 
ence  of  hypothesis  Hq  or  hypothesis  Hj  as  determined  by  the  local  detector  LD,.  In  the 
second  case,  the  observation  space  of  each  observation  Xj  is  partitioned  into  M  nonover¬ 
lapping  regions.  The  local  decision  Zj,  correspondingly,  takes  on  one  of  M  possible  values 
depending  upon  the  region  in  which  the  observation  Xj  falls  in.  In  this  case,  the  local  deci¬ 
sion  Zj  does  not  contain  explicit  information  about  the  hypothesis  present.  In  both  the  hard 
decision  and  the  soft  decision  cases,  the  output  alphabet  is  finite  and  each  element  in  this 
alphabet  has  a  finite  probability  of  occurrence.  This  means  that  the  entropy  of  the  alphabet 
is  finite  and,  therefore,  a  finite  capacity  channel  can  be  employed.  The  local  decisions 
(hard  or  soft)  are  sent  over  the  bandlimited  channels  to  the  global  decision  maker,  which  is 
also  referred  to  as  the  fusion  center.  Based  on  the  decision  vector  U=[zj,  Z2,...,z„]  whose 
elements  are  the  decisions  made  by  the  individual  sensors,  the  fusion  center  makes  the  fi¬ 
nal  decision  on  whether  hypothesis  Hq  or  hypothesis  Hj  is  true. 

Figure  1.2  shows  a  variation  of  the  parallel  fusion  system  shown  in  Figure  1.1.  Here, 
the  global  decision  maker  receives  a  local  observation  of  its  own  (side  information)  in  ad¬ 
dition  to  the  local  decisions  Zj,  i=l,2,...,n.  The  observation  vector  based  on  which  the  fu¬ 
sion  center  makes  the  final  decision  is  the  augmented  vector  QJXo]  of  the  local  decisions 
24  and  the  observation  Xq  at  the  fusion  center.  Intuitively,  the  performance  of  this  system  is 
expected  to  be  better  than  the  performance  of  the  parallel  fusion  system  with  n  local  detec¬ 
tors.  Here  the  information  in  the  observation  Xq  is  fully  utilized  by  the  fusion  center  and  is 
not  compressed  by  a  local  detector.  In  the  interesting  special  case  when  n  =1,  the  system 
reduces  to  a  two-stage  decentralized  serial  configuration. 

Figure  1.3  shows  a  hierarchical  system  with  local  and  regional  decision  makers.  The 
system  consists  of  2n  local  decision  makers,  n  regional  decision  makers  (RD’s)  and  a  glo¬ 
bal  decision  maker.  Local  detectors  LX>2i-i  and  LD2j  process  their  locally  received  obser¬ 
vations  X2j.j  and  X2j  and  forward  their  decisions  z2j_i  and  Z2j  to  an  intermediate  regional 
decision  maker  RDj ,  i=l,...,n.  The  regional  detector  RDj  combines  the  two  local  deci¬ 
sions  along  with  its  directly  received  observation  Yj  to  make  the  regional  decision  Uj.  The 
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decision  vector  U=[uj...un]  is  used  by  the  global  decision  maker  to  make  the  final  decision 
uq.  The  case  of  more  than  two  local  detectors  per  regional  decision  maker  can  also  be  con¬ 
sidered  but  is  not  treated  in  this  dissertation.  Figure  1.4  shows  another  hierarchical  decen¬ 
tralized  system.  The  difference  between  this  system  and  the  one  shown  in  Figure  1.3  is 
that  the  regional  decision  makers  receive  no  observations  of  their  own.  Therefore,  they 
have  to  make  their  decisions  solely  on  the  basis  of  the  local  decisions  they  receive.  The  lo¬ 
cal  decisions  are  made  on  the  basis  of  the  local  observations  and  the  global  decision  is 
made  on  the  basis  of  the  decisions  received  from  the  regional  decision  makers.  However, 
due  to  the  unavailability  of  observations  at  the  regional  detectors,  the  performance  of  this 
system  is  expected  to  be  inferior  to  the  performance  of  the  hierarchical  system  with  side 
information  at  the  regional  level. 

1.2  Literature  Survey 

In  this  section,  we  briefly  review  related  work  on  some  topics  that  are  treated  in  this 
dissertation. 

1.2.1  Decentralized  Detection  Systems 

The  design  of  decentralized  detection  systems  has  been  dealt  with  by  a  number  of 
authors,  e.g.  [4-29].  The  parallel  fusion  system,  in  particular,  has  gained  most  of  the  inter¬ 
est  in  the  literature.  Tenney  and  Sandell  [5]  addressed  the  problem  of  structuring  a  set  of 
decision  makers  and  communication  links  in  a  manner  which  leads  to  effective  manage¬ 
ment  of  a  complex,  large  scale  system  in  real  time.  In  a  follow  up  paper  [6],  they  devel¬ 
oped  mechanisms  based  on  the  interactions  between  subsystems  to  coordinate  the  making 
of  decisions  which  are  “best”  in  some  system-wide  sense.  One  interesting  application  of 
their  work  is  the  area  of  distributed  detection  systems.  Tenney  and  Sandell  [7]  treated  the 
two-sensor  decentralized  detection  problem  with  no  data  fusion  from  a  Bayesian  point  of 
view.  Costs  were  assigned  to  reflect  the  course  of  action  of  each  local  detector.  The  local 
decision  rules  were  chosen  such  that  the  average  cost  is  minimized.  The  local  decision 
rules  were  shown  to  be  likelihood  ratio  tests  of  the  sensor  observations  for  conditionally 
independent  observations.  Sadjadi  [8]  extended  the  theory  in  [7]  to  encompass  the  M  hy- 
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pothesis  testing  problem  with  n  local  detectors.  Here  again,  the  fusion  center  was  not  a 
part  of  the  optimization.  Assuming  known  sensor  thresholds,  Chair  and  Varshney  [9]  de¬ 
veloped  a  minimum  average  cost  algorithm  for  combining  the  sensor  decisions  in  an  n- 
sensor  system  at  the  fusion  center.  Optimization  of  the  entire  system  was  considered  by 
Hoballah  and  Varshney  [10]  where  they  obtained  a  person- by-person  optimal  solution  to 
the  parallel  network  with  n  local  detectors  and  a  fusion  center.  The  person-by-person  opti¬ 
mal  design  for  the  binary  hypothesis  testing  problem  requires  .the  joint  solution  of  (2n+n) 
simultaneous  nonlinear  equations.  An  iterative  algorithm  for  the  solution  of  the  person- 
by-person  optimality  equations  was  proposed  in  [11].  This  algorithm  is  based  on  the 
Gauss-Siedel  method  for  the  solution  of  coupled  nonlinear  equations.  The  optimization  of 
the  entire  system  was  also  considered  by  Reibman  and  Nolte  [12]  where  an  exhaustive 
search  is  performed  over  all  the  fusion  rules  in  order  to  determine  the  overall  minimum 
cost  solution.  In  the  exhaustive  search  method,  the  fusion  center  is  fixed  at  a  particular  fu¬ 
sion  rule  and  a  set  of  n  coupled  nonlinear  equations  are  solved  to  determine  the  n  local 
thresholds.  This  process  has  to  be  repeated  for  all  the  permissible  fusion  rules.  The  fusion 
rule  along  with  the  n  local  thresholds  that  yield  the  smallest  possible  cost  are  the  optimum 
system  design  parameters.  The  number  of  fusion  rules  grows  quite  rapidly  with  n.  Tho- 
mopoulos,  Viswanathan,  and  Bougoulias  [13]  showed  that  the  number  of  fusion  rules  to 
be  examined  for  various  values  of  n  is  given  as  shown  in  the  following  table. 

Number  of  local  detectors  Number  of  permissible  fusion  rules 

2  2 

3  9 

4  114 

5  6894 
Table  1.1 

In  both  the  exhaustive  method  and  the  person-by-person  optimization  approach,  the 
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amount  of  computation  required  grows  exponentially.  This  makes  the  above  design  proce¬ 
dures  computationally  difficult  and,  therefore.the  study  of  decentralized  detection  systems 
has  been  limited  to  small  networks  and  very  few  topologies.  In  this  dissertation,  we 
present  a  computationally  simpler  approach  for  the  design  of  decentralized  Bayesian  de¬ 
tection  systems.  Using  this  approach,  we  show  that  the  design  of  the  optimum  decentral¬ 
ized  parallel  fusion  system  reduces  to  the  optimization  of  a  single  function  of  n  variables. 
Chair  and  Varshney  [14]  considered  the  problem  of  distributed  Bayesian  hypothesis  test¬ 
ing  with  distributed  data  fusion  in  which  data  fusion  is  performed  at  each  site. 

The  Neyman  -Pearson  criterion  has  also  been  used  in  the  design  of  decentralized  de¬ 
tection  systems  where  neither  the  costs  nor  the  a  priori  probabilities  need  to  be  available  to 
the  designer.  Srinivasan  [15]  used  the  Neyman-Pearson  criterion  to  obtain  the  local  deci¬ 
sion  rules  in  the  parallel  fusion  network  assuming  that  the  fusion  center  is  a  combinational 
logic  circuit.  Hoballah  and  Varshney  [16]  treated  the  problem  in  two  respects.  First,  when 
the  fusion  rule  is  known  and  the  objective  is  to  find  the  local  decision  rules.  Second,  when 
the  decision  rules  at  the  detectors  are  given,  and  the  objective  is  to  find  the  optimum  fusion 
rule.  Thomopoulos,  Viswanathan,  and  Bougoulias  [17]  employed  the  Neyman-Pearson 
criterion  where  both  the  decision  made  by  each  individual  sensor  and  the  global  decision 
made  by  the  fusion  center  aie  based  on  the  Neyman-Pearson  test.  Tsitsiklis  [18]  addressed 
the  question  of  concavity  of  the  receiver  operating  characteristic  of  the  system.  He  found 
that  for  a  given  strategy,  the  receiver  operating  characteristic  is  not  necessarily  a  concave 
function  ( a  numerical  example  is  provided  in  [19] ).  However,  concavity  can  be  achieved 
by  randomizing  with  respect  to  the  possible  strategies.  A  similar  result  was  obtained  by 
Willet  and  Warren  [20],  Viswanathan,  Thomopoulos,  and  Tumuluri  [21]  applied  the  Ney¬ 
man-Pearson  criterion  to  the  design  of  the  serial  decentralized  configuration.  They  found 
that  for  the  case  of  two  sensors,  the  optimal  serial  network  has  a  better  performance  than 
the  parallel  scheme  ( better  here  refers  to  higher  probability  of  detection  for  the  same  false 
alarm  probability).  While  this  interesting  result  is  true  for  the  case  of  two  sensors,  the  nu¬ 
merical  examples  provided  by  [21]  show  that  this  result  is  not  true,  in  general,  for  systems 
with  more  than  two  sensors. 
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The  Bayesian  formulation  has  also  been  applied  to  the  design  of  decentralized  detec¬ 
tion  structures  other  than  the  parallel  fusion  network.  Ekchian  and  Tenney  [22]  derived  the 
necessary  conditions  for  the  thresholds  to  satisfy  in  order  to  minimize  a  given  cost  func¬ 
tion  for  a  number  of  configurations  including  the  tandem  and  the  tree-hierarchical  topolo¬ 
gies.  No  numerical  results  were  provided.  Reibman  and  Nolte  [23]  applied  the  exhaustive 
search  method  to  design  specific  decentralized  detection  configurations.  Tsitsiklis  [24]  and 
Varshney  [25]  provided  an  overview  of  the  recent  advances  in  the  theory  of  decentralized 
detection  systems. 

1.2.2  The  Class  of  Ali-Silvey  Distance  Measures  and  the  Detection 
Problem 

The  class  of  Ali-Silvey  distance  measures  has  played  an  important  role  in  the  design 
of  quantizers  for  hypothesis  testing.  Recently,  it  has  also  been  employed  in  the  design  of 
decentralized  detection  systems.  In  this  dissertation,  we  apply  members  of  the  class  of  Ali- 
Silvey  distance  measures  to  design  suboptimum  decentralized  detection  systems,  and 
compare  their  performance  to  that  of  the  optimum  decentralized  detection  system.  For  the 
sake  of  completeness  we  now  define  the  class  of  Ali-Silvey  distance  measures  and  provide 
examples  of  members  of  this  class. 

Let  Pq(x)  and  pj(x)  be  the  conditional  probability  density  functions  of  the  random 
variable  X  under  hypotheses  Hq  and  Hj  respectively.  A  measure  of  dissimilarity  between 
these  density  functions  can  be  expressed  in  terms  of  the  general  class  of  Ali-Silvey  dis¬ 
tance  measures  (or  the  class  of  f-divergences)  which  is  defined  as  [30] 

D  =/{E0[g(/)]}  (1.1) 

where 

Eq  is  the  expectation  under  the  hypothesis  Hq 

g  is  a  convex  real-valued  function  defined  on  (0,°°) 
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f  is  an  increasing  function 
l  =  pj(x)  /  po(x)  is  the  likelihood  ratio  function 


Examples  of  measures  from  this  class  include 
1)  The  Bhattacharyya  distance 


db  ~  ~lnjJPo  MP  i  (*) dx 

X 


2)  The  discrimination  distance  measure 


r  P  oM 

Dd  =  jp0(x)ln—dx 


(1.2) 


(1-3) 


3)  The  Kolmogorov  variational  distance 

dk  =  JlPiU)  ~PoM\dx  (1.4) 

X 


4)  The  J-divergence 


f  Pi  U) 

Dj  =  J  [pi  (x)  -p 0  (Jt)  ]  ln-^-jdx 


(1.5) 


The  class  of  Ali-Silvey  distance  measures  has  received  an  increasing  interest  in  the 
design  of  quantizers  for  hypothesis  testing.  This  is  due  to  the  strong  link  between  these 
distance  measures  and  the  probability  of  error  (POE).  Let  tt  =  {7tQ,  Jti }  be  the  set  of  all 
permissible  pairs  of  the  prior  probabilities  in  a  binary  hypothesis  testing  problem.  Then, 
there  exists  a  subset  of  7t  for  which  if  the  distance  between  a  given  set  of  conditional 
densities  is  larger  than  the  distance  between  another  set  of  conditional  densities,  then  the 
POE  corresponding  to  the  first  set  is  less  than  the  POE  corresponding  to  the  second  set. 
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This  result  is  known  as  the  Blackwell  theorem  [31].  Poor  and  Thomas  [32]  applied  the 
general  class  of  Ali-Silvey  distance  measures  to  the  design  of  a  generalized  quantizer  for 
binary  decision  systems.  Poor  [33]  also  used  these  measures  as  criteria  for  analyzing  the 
effects  of  fine  data  quantization  on  inferential  procedures  and  for  designing  quantizers  to 
minimize  these  detrimental  effects.  Benitz  and  Bucklew  [34]  applied  the  Chemoff 
theorem  to  the  design  of  quantizers  that  asymptotically  minimize  the  probability  of  error 
(The  Chemoff  theorem  differs  from  the  traditional  Chemoff  bound  in  that  the  theorem 
clearly  exhibits  the  exponential  dependence  of  the  bound).  A  number  of  authors  have 
applied  members  of  the  class  of  Ali-Silvey  distance  measures  in  the  design  of 
decentralized  detection  systems.  For  example,  Longo,  Lookabaugh,  and  Gray  [35]  have 
employed  the  Bhattacharyya  distance,  whereas  Lee  and  Chao  [36]  have  used  the  J- 
divergence  to  subpartition  the  decision  space  when  a  quality  bit  is  transmitted  along  with 
the  decision  to  the  fusion  center. 

1.23  Information  Theoretic  Measures  for  Quantization  and  Detection 

The  quantization  of  a  random  variable  X  for  minimum  distortion  has  been  studied 
extensively  in  the  literature  (see  [39]  for  a  survey  of  results).  When  the  entropy  of  the 
quantizer  output  is  restricted  not  to  exceed  a  given  prescribed  value,  the  problem  becomes 
that  of  quantization  under  entropy  constraint.  The  quantization  of  the  random  variable  X 
for  minimum  distortion  under  entropy  constraint  was  considered  by  Noll  and  Zelinski  [40] 
and  Farvardin  and  Modestino  [41].  It  is  known  that  in  detection  and  estimation  problems, 
quannzation  for  minimum  distortion  is  not  the  appropriate  criterion  to  use  for  designing  an 
optimum  inferential  system  [42,43].  The  problem  of  adjusting  the  threshold  in  a  simple  bi¬ 
nary  hypothesis  testing  problem  under  the  condition  of  maximizing  the  mutual  informa¬ 
tion  between  the  decision  and  the  state  of  nature  was  considered  by  Middleton  [44]  and 
Gabriele  [45].  Martinez  [46]  and  Hoballah  and  Varshney  [47]  have  shown  that  the  prob¬ 
lem  of  maximizing  the  mutual  information  between  the  decision  and  the  state  of  nature  is 
equivalent  to  applying  the  Neyman-Pearson  criterion  for  signal  detection.  As  a  result,  the 
maximum  mutual  information  detector  is  a  likelihood  ratio  detector.  Hoballah  and  Varsh- 


13 


ney  [47]  have  made  use  of  this  result  to  design  a  maximum  mutual  information  decentral¬ 
ized  detector  under  the  assumption  that  each  local  detector  makes  a  single  hard  decision. 

1.2.4  Bounds  on  the  Probability  of  Error  of  Optimum  Receivers 

In  hypothesis  testing,  the  performance  of  optimum  receivers  is  usually  expressed  in 
terms  of  the  probability  of  error.  It  is  well  known  that  the  optimum  receiver  which  mini¬ 
mizes  the  probability  of  error  is  the  maximum  a  posteriori  probability  (MAP)  receiver  [3]. 
When  an  observation  x  is  received,  the  MAP  receiver  computes  the  a  posteriori  probabili¬ 
ties  P(Ho  I  x)  and  P(Hj  I  x)  of  the  two  hypotheses  and  chooses  the  hypothesis  with  the  larg¬ 
er  a  posteriori  probability.  The  probability  of  making  an  error  based  on  the  observation  x  is 
given  as 

P  (Error\ x)  =  min(P  (H0\x),P  (H^x))  (1.6) 

Analytic  evaluation  of  the  probability  of  error  is  very  difficult  in  most  applications  be¬ 
cause  it  involves  the  evaluation  of  the  discontinuous  function  min  (.).  Instead  of  evaluat¬ 
ing  the  exact  probability  of  error,  tight  upper  and  /  or  lower  bounds  can  often  be 
determined  analytically  making  it  possible  to  compare  the  performance  of  optimum  re¬ 
ceivers  based  on  these  bounds. 

A  number  of  upper  and  lower  bounds  have  been  proposed  in  the  literature  [51-63]. 
Because  of  the  indirect  relationship  between  the  probability  of  eiTor  and  the  class  of  Ali- 
Silvey  distance  measures  discussed  above,  a  number  of  the  bounds  available  are  expressed 
in  terms  of  these  distance  measures.  The  idea  behind  this  lies,  of  course,  in  the  Blackwell 
theorem.  Boekee  and  Van  der  Lubbe  [51]  provided  upper  bounds  on  the  probability  of  er¬ 
ror  by  considering  the  f-divergence  between  the  conditional  densities  under  the  two  hy¬ 
potheses.  They  have  shown  that  this  upper  bound  includes  many  well  known  bounds  in 
terms  of  other  distance  measures.  The  Bhattacharyya  bound  [31]  is  the  simplest  bound  to 
evaluate.  Its  simplicity  and  the  fact  that  closed  form  expressions  for  the  bound  exist  for 
many  commonly  used  distributions,  made  it  an  attractive  tool.  The  most  common  applica- 


tions  are  the  design  of  quantizers  for  hypothesis  testing  [32,35]  and  signal  selection  [31]. 
The  main  disadvantage  of  the  Bhattacharyya  bound  is  that  it  is  a  loose  bound.  The 
Chemoff  bound  [52,53]  provides  an  upper  bound  on  the  probability  of  error  in  terms  of  a 
scalar  s,  0<s<l.  The  tightest  bound  is  obtained  by  optimizing  the  upper  bound  with  respect 
to  the  scalar  s.  The  difficulty  in  evaluating  the  Chemoff  bound  makes  it  less  attractive  than 
the  Bhattacharyya  bound.  In  addition,  the  Chemoff  bound  which  reduces  to  the  Bhatta¬ 
charyya  bound  when  s=0.5  does  not,  in  general,  provide  tighter  error  bounds  than  the 
Bhattacharyya  bound  [54].  A  tighter  bound  on  the  probability  of  error  than  the  Bhatta¬ 
charyya  bound  is  in  terms  of  the  equivocation  function  [55,53].  Devijver  [56]  introduced 
another  bound  in  terms  of  the  so  called  Bayesian  distance.  This  bound  is  known  to  be 
tighter  than  both  the  Bhattacharyya  bound  and  the  equivocation  bound.  As  applied  to  de¬ 
centralized  systems,  lew  results  are  available  in  the  literature.  Kazakos  [57]  employed  the 
concept  of  distance  measures  to  obtain  bounds  on  the  performance  of  distributed  detection 
systems.  Tsitsiklis  [58]  considered  the  decentralized  problem  when  the  number  of  sensors 
tends  to  infinity.  He  showed  that  it ’s  asymptotically  optimal  for  the  sensors  to  use  the 
same  decision  rule  if  they  are  operating  under  identical  circumstances. 

1.3  Dissertation  Outline 

In  this  dissertation,  we  consider  some  design  and  analysis  aspects  of  a  number  of 
decentralized  detection  structures.  Our  main  focus  is  on  the  Bayesian  approach  to  the  de¬ 
sign  of  these  systems.  The  design  of  these  systems  involves  the  design  of  local  and  global 
decision  rules.  The  design  of  optimum  decentralized  detection  systems  based  on  the  Baye¬ 
sian  formulation  is  considered  in  detail.  The  performance  of  the  optimum  systems  is  com¬ 
pared  to  the  performance  of  suboptimum  systems  designed  based  on  criteria  other  than  the 
global  optimum  Bayesian  cost.  Upper  and  lower  bounds  on  the  minimum  probability  of 
error  and  the  minimum  achievable  cost  are  derived  for  the  conventional  centralized  detec¬ 
tion  system.  A  new  tight  upper  bound  on  the  minimum  probability  of  error  is  presented 
and  applied  to  design  a  nearly  optimum  decentralized  detection  system.  The  role  of  ran¬ 
domization  of  the  global  decision  rule  in  the  Bayesian  problem  and  in  the  related  Neyman- 
Pearson  problem  is  also  discussed. 
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In  Chapter  2,  we  study  the  problem  of  optimum  receivers  from  a  Beyesian  view¬ 
point.  We  derive  an  alternate  representation  of  the  minimum  achievable  cost  of  an  opti¬ 
mum  receiver  in  terms  of  a  modified  form  of  the  Kolmogorov  variational  distance.  Using 
this  representation,  we  show  that  randomization  of  the  decision  rule  is  not  necessary  in  the 
case  when  the  observations  assume  discrete  values. 

In  Chapter  3,  we  generalize  some  known  upper  and  lower  bounds  on  the  minimum 
probability  of  error  to  the  general  Bayesian  problem  ,i.e.,  we  obtain  upper  and  lower 
bounds  on  the  minimum  average  cost  of  optimum  Bayesian  receivers.  These  bounds  in¬ 
clude  the  Chemoff  bound  and  the  Bhattacharyya  bound.  We  derive  a  new  tight  upper 
bound  on  the  minimum  probability  of  error  and  generalize  it  to  the  general  Bayesian  prob¬ 
lem.  New  tight  lower  bounds  on  the  probability  of  error  and  minimum  average  cost  of  an 
optimum  receiver  arc  also  obtained. 

In  Chapter  4,  we  consider  the  analysis  and  the  design  of  the  parallel  fusion  system 
from  a  Bayesian  point  of  view  assuming  identical  local  detectors.  The  analysis  and  the  de¬ 
sign  of  the  system  are  based  on  the  representation  of  the  minimum  achievable  cost  of  opti¬ 
mum  receivers  derived  in  Chapter  2.  Both  the  hard  and  the  soft  decision  cases  are 
considered.  Using  this  representation  we  show  that  the  design  of  the  optimum  decentral¬ 
ized  detection  system  reduces  to  the  optimization  of  a  single  function  of  a  given  number  of 
variables  that  depend  upon  the  number  of  quantization  levels.  The  design  of  suboptimum 
decentralized  detection  systems  based  upon  members  of  the  class  of  Ali-Silvey  distance 
measures  are  also  considered.  The  performance  of  these  systems  are  compared  to  the  per¬ 
formance  of  the  optimum  system.  The  design  of  the  decentralized  detection  system  based 
on  the  new  upper  bound  on  the  minimum  probability  of  error  is  also  discussed  . 

In  Chapter  5,  we  consider  the  design  of  the  four  decentralized  detection  structures 
shown  in  Figures  1.1  -  1.4  based  on  the  Bayesian  formulation.  The  local  detectors  are  as¬ 
sumed  to  yield  hard  decisions  but  are  not  assumed  to  be  identical.  The  optimum  design  of 
each  configuration  is  shown  to  reduce  to  the  optimization  of  a  single  function  of  a  given 
number  of  variables  depending  upon  the  configuration. 

In  Chapter  6,  we  examine  the  problem  of  randomization  of  the  decision  rule  for  de- 
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tection  systems  designed  based  on  the  Neyman-Pearson  criterion.  When  neither  the  a  prio¬ 
ri  probabilities  nor  the  costs  are  known,  the  Neyman-Pearson  criterion  becomes  useful. 
We  show  that  when  the  objective  is  to  design  a  decentralized  detection  system  that  maxi¬ 
mizes  the  global  probability  of  detection  for  a  given  global  false  alarm  probability,  ran¬ 
domization  of  the  decision  rule  at  the  fusion  center  is  not  necessary  and,  in  fact,  if  used 
deteriorates  the  system  performance. 

In  Chapter  7  we  present  a  summary  of  the  results  obtained  in  this  dissertation  plus 
some  concluding  remarks.  We  also  discuss  some  of  the  problems  related  to  the  topics  dis¬ 
cussed  in  the  dissertation  that  need  to  be  addressed  in  the  future. 
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CHAPTER  2 

MINIMUM  AVERAGE  COST  RECEIVERS 


2.1  Introduction 

As  indicated  in  Chapter  1,  one  of  the  objectives  of  this  dissertation  is  to  consider  the 
design  of  decentralized  Bayesian  detection  systems.  In  order  to  facilitate  the  design  of 
these  systems  we  first  consider  centralized  Bayesian  detection  systems,  i.e.,  the  systems  in 
which  all  the  raw  observations  are  processed  at  one  central  location.  A  system  (centralized 
or  decentralized)  that  minimizes  the  Bayesian  cost  will  be  referred  to  as  a  minimum  aver¬ 
age  cost  (MAC)  receiver.  In  this  chapter  we  will  derive  some  important  properties  of  the 
classical  centralized  optimum  receiver.  These  are  generalizations  of  the  results  available  in 
the  literature  for  the  minimum  probability  of  error  receiver  [3 1 ,37] .  These  results  will  then 
be  applied  to  the  design  and  performance  evaluation  of  decentralized  detection  structures 
in  the  following  chapters. 

Let  fo(x)  and  f\{x)  be  the  conditional  probability  density  functions  of  the  random  ob¬ 
servations  X  under  the  two  hypotheses  Hg  and  Hi  to  be  tested  at  the  receiver.  Also,  let  7t0 
and  it]  be  the  corresponding  a  priori  probabilities  of  Hg  and  Hj  respectively.  If  Cl} ;  i,j= 0, 
1 ,  denotes  the  cost  of  deciding  Hj  when  Hj  is  true,  then  the  average  cost  per  decision  made 
by  the  receiver  is  given  by 

l  l 

*  =  I  I  CijPiHj'H;)  (2.1) 

i  =  0j  =  0 

where  P(Hj,  Hj)  is  the  probability  of  the  joint  event  that  Hj  is  true  and  Hj  is  decided.  It  is 
well  known  [1]  that  the  cost  given  by  (2.1)  is  minimized  when  the  following  decision  rule 
is  used 
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(2.2) 


Hi 

f\(x)  > 

/«W  <  71 1  ((2qi  —  Cu)  ^ 


Ho 

This  decision  rule  partitions  the  observation  space  of  X  into  two  optimum  decision  regions 
Zk*;  k  =  0,  1,  such  that  when  x  e  Zk*.  Hk  is  declared  true.  The  minimum  average  cost  can 
be  determined  from  (2.1)  using  the  optimum  decision  regions  Zk*. 


2.2  An  Alternate  Representation  for  the  Minimum  Average  Cost 

To  present  our  approach  for  the  design  of  minimum  average  cost  (MAC)  receivers, 
we  express  the  optimum  decision  rule  (2.2)  and  the  corresponding  minimum  average  cost 
given  in  (2.1)  in  an  alternate  way.  We  begin  by  expressing  (2.1)  as 

i  l 

<2-3> 

i  =  0j=0 


where  P  (Hj  I  Hj)  is  the  conditional  probability  of  deciding  H;  when  Hj  is  true.  Let  Zk  ,  k  = 
0,  1,  be  the  decision  region  corresponding  tr  Hk  .  su.h  that  when  the  observation  x  E  Zk, 
Hk  is  declared  true.  The  a  rosteric.i  probabilities  P  (Hj  I  Hj)  are  given  as 

=  jfj(x)dx  (2.4) 

z, 


Sr'  .tuting  (2.4)  into  (2.3)  we  get 


R  = 


l  l 


I  I  CiftjfjWdx 


(2.5) 


Using  Bayes  rule,  we  have 


*fj(x)  =  P(Hj\x)f(x) 

X, 

vx 

X 


(2.6) 
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where 


fix)  =  7t/0(x)  +71/!  (x) 


is  the  unconditional  density  function  of  X.  Using  (2.6),  we  express  (2.5)  as 


(2.7) 


l  l 

«  =  0/  =  0  Z; 


(2.8) 


Since  Qj  is  a  constant,  we  interchange  the  order  of  integration  and  the  inner  summation  as 

l  l 


R  =  *)/(*) 
;  =  ozj  =  o 


(2.9) 


Expanding  (2.9)  over  the  outer  summation,  we  get 


R  =  Jp0(x)/(x)dx  +  ||31(x)/(x)dx 
Zo  Zi 


(2.10) 


where 


p0(x)  =  C00/’(//0|x)+C01/>(//1|x) 


P/x)  =  C10P(//0jx)  +Cil/>(//1jx) 


(2.11) 


(2.12) 


The  result  in  (2.10)  reveals  that  the  average  cost  given  by  (2.1)  can,  equivalently,  be  repre¬ 
sented  as  the  expected  value  of  the  random  variable  P(x)  defined  as 
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Po(x)  ifxeZo 

PW  =  \  (2.13) 

^  3l(x)  if  x  £  Z\ 

The  quantities  Pq(x)  anti  J3i(x)  represent  the  conditional  costs  assigned  to  each  point  x  in 
the  observation  space.  When  an  observation  x  is  received,  the  receiver  computes  the  con¬ 
ditional  costs  f3o(x)  and  J3j(x).  If  the  objective  at  the  receiver  is  to  minimize  the  cost,  then 
it  should  select  the  hypothesis  with  the  smaller  conditional  cost.  The  optimum  decision 
rule  (2.2)  based  on  the  conditional  cost  formulation  becomes 

Hi 

*  Pi  to  (2-14) 

Ho 

Denote  by  r(x)  the  conditional  cost  of  the  MAC  receiver  based  on  an  observation  x.  Then 
r(x)  can  be  expressed  as 

r(x)  =  min(p0(x),  (3,  (x))  '  (2.15) 

Using  the  mathematical  identity 

min  (a,  b)  =  1  (a  +  b)  -  j\a-b\  (2.16) 

we  can  rewrite  (2.15)  as 

r(x)  =  ^  [P0W  +pj  (*)]  -^|p0(x)  -Pj  (or) |  (2.17) 

Substituting  (2.11)  and  (2.12)  into  (2.17)  we  get 
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r(x)  =  l[C00l»(H0|*)+C,0f(ff0|i)+C01/’(Hl|*)+C11P(//1|*)] 

-jICqoP  («»| *>  -  Cl0P  (Ho|  x)  +  CmP  (H, |jc)  -  CnP  (H, |jr)  |  (2.18) 

The  minimum  average  cost  Rm  is  the  expected  value  of  the  conditional  cost  r(x),  that  is 

Rm  ~  jr (*)  [Vo  (*)  +  TCj/i  U)  1  dx  (2.19) 

For  notational  simplicity,  the  subscript  m  will  be  omitted  from  Rm  in  the  rest  of  this  disser¬ 
tation.  Using  (2.6),  we  can  evaluate  the  a  posteriori  probabilities  as 

nfj  (x) 

PiHAx)  =  JTxT  (2-20) 

Substituting  (2.20)  into  (2.18)  we  obtain 

r  =  2 f(x)  ^  ^0®  +  Cio)  Vo  (•*)  +  (Coi  +  ^n) 71 1 f i  (x)  ] 

~2f(x)  I  ^°°  ~  7t°^°  ^  +  ^0I  ~  ^'1I^  1  (2.21) 

Substituting  (2.21)  into  (2. 19)  and  integrating  term  by  term  we  obtain  the  following  ex¬ 
pression  for  the  minimum  average  cost  R 

R  =  Ro~\\\c\fi  (x)  -Cf/0(x)\dx  (2.22) 
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where 


=  2  (Coo  +  C 10)  +  2  ^01  +  ) 71 1 

Ci  =  (^oi  ~ ^n)  Jlj 

Cq  -  (C10  -  Cqq)  Tig 

When  the  random  variable  X  assumes  discrete  values,  the  MAC  ,  R,  becomes 

R  =  *0  -  \  S|C^  (X=  ij  «,)  -  C0f>  (X=  JC,-|  H„)  |  (2.23) 

4//  » 

2.3  Randomization  in  Minimum  Average  Cost  Receivers 

When  the  random  variable  X  assumes  continuous  values,  the  contribution  to  the 
minimum  average  cost  of  those  values  of  X  that  satisfy  equality  in  the  decision  rule  (2. 14) 
is  zero.  This  is  because  these  values  occur  with  zero  probability.  Therefore,  they  can  be  as¬ 
signed  to  either  one  of  the  decision  regions  without  affecting  the  minimum  average  cost. 
When  X  is  discrete,  the  values  of  X  that  satisfy  equality  in  (2.14)  occur  with  finite  proba¬ 
bility.  Therefore,  randomization  of  the  decision  rule  may  become  necessary.  While  parti¬ 
tioning  the  observation  space  of  the  discrete  random  variable  X  into  the  decision  regions 
corresponding  to  H q  and  Hj,  the  receiver  uses  the  decision  rule  (2.2)  with 

P(X=  xJZ/j) 

A(JC,)  “  P  (X=  3cJ  H0)  (224) 

Let  K={ki,k2,...,k/}  be  the  set  of  those  values  of  X  that  satisfy  the  relation 
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P(X=  _  *0(Cl0-Cm) 

P(X=  ^//0)  Hj(C01-Cn) 


(2.25) 


The  complement  set  Kc  represents  all  the  remaining  values  that  the  random  variable  X  as¬ 
sumes.  Using  Bayes  rule  we  can  express  (2.25)  as 


P(//1|X=  k.)  _  (C10- Cqq) 
P(H0X=  kt)  (C01-Cn) 


(2.26) 


The  conditional  costs  Pq(x)  and  Pi(x)  for  the  outcomes  that  satisfy  (2.26)  are  given  by 


M*<)  =  coop(Ho\X=  *,-)  +C0l  P(H,\ X=  k{) 


(2.27) 


P^i)  =  Cl0P(H0\X=  ki)  +CnP(Hl  |X=  *,) 


(2.28) 


Using  (2.26)  we  get 


P{HxX=  kr)  =  P(H0  \X=  kt) 


(C10 

(C01  -  Cn) 


(2.29) 


Substituting  (2.29)  into  (2.27)  and  (2.28) ,  and  simplifying  we  get 


(CinCni  —  C  iiC  nr\) 

P0 (*.)  =  P,(*/>  =  P(H0\X=  *,)  V  _r  ■  e  K 

^01  LI1 


(2.30) 


The  result  in  (2.30)  indicates  that  such  boundary  outcomes  are  equally  costly  and  can  be 
assigned  arbitrarily  to  any  one  of  the  decision  regions.  Using  (2.23)  we  will  show  that  the 
contribution  of  these  outcomes  to  the  MAC  is  zero.  This  can  be  done  by  expanding  (2.23) 
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,  ' 

*  =  ^o-jSFi  P(X=  kl\Hl)-C0P(X=  ki  |//0)| 

<■=  1 

-i  X  F,/>(X=  xj//,)  -C0/>(X=  x,\H0)\  (2.31) 

x;e  Afe 

In  (2.31),  the  second  term  on  the  right  hand  side  is  the  sum  over  all  those  values  of  X  that 
are  elements  of  K,  and  the  third  term  is  the  sum  over  all  the  values  that  are  elements  of  Kc. 
Using  (2.25)  we  get 

i 

£|C,/>(X=  k,\H,)  -C0P  (X=  *Jtf0)|  =  0  (2.32) 

i-  1 

The  result  in  (2.32)  shows  that  it  is,  in  fact,  immaterial  as  to  whether  the  outcomes  that  sat¬ 
isfy  (2.25)  are  assigned  to  the  decision  space  of  Hq  or  to  the  decision  space  of  Hj  since 
their  contribution  to  the  MAC  is  zero.  Thus,  randomization  is  not  necessary.  Therefore, 
whether  the  random  variable  X  is  continuous  or  discrete,  the  minimum  average  cost  is  not 
affected  by  those  values  of  X  that  satisfy  equality  in  the  likelihood  ratio  test. 

2.4  Minimum  Probability  of  Error  Receivers 

As  an  important  application  of  the  general  Bayesian  formulation  developed  in  Sec¬ 
tion  2.2,  we  consider  the  special  cost  assignment  Cqo  =  Cu  =  0  and  Cqj  =  Cjq  =  1.  This 
corresponds  to  the  minimum  probability  of  error  criterion  which  is  widely  used.  In  this 
case,  the  conditional  costs  (3q(x)  and  Pj(x)  given  by  (2.11)  and  (2.12)  become 

P0(jc)  =  P{Hx\x)  (2.33) 


Pj  U)  =  P(H0\x)  (2.34) 

and  the  decision  rule  (2.14)  becomes 

Hi 

PiH^x)  >  P(H0 \x)  (2.35) 

Ho 

Decision  rule  (2.35)  indicates  that  the  MAC  receiver  reduces  to  a  maximum  a  posteriori 
probability  (MAP)  receiver  in  this  special  case.  When  an  observation  x  is  received,  the  re¬ 
ceiver  computes  the  a  posteriori  probabilities  P  (HqIx)  and  P  (Hjlx)  and  chooses  the  hy¬ 
pothesis  with  the  larger  a  posteriori  probability.  The  function  r(x)  defined  in  (2.15) 
representing  the  conditional  cost  of  the  MAC  receiver  based  on  an  observation  x  becomes 

r(x)  =  min{P  (Hq\x),P  (Hx\x))  (2.36) 


This  function  now  defines  the  probability  that  the  MAP  receiver  makes  an  error  based  on 
an  observation  x.  The  minimum  probability  of  error  (MPOE)  is  obtained  from  (2.22)  and 
is  given  by 


p(£)  =  \  ~  (*>  "  Vo  (*)|«k 

X 


(2.37) 


which  is  the  same  expression  as  obtained  by  Kailath  [31]  and  Toussaint  [37]  using  a  differ¬ 
ent  approach. 

In  this  chapter,  we  have  obtained  an  expression  for  the  minimum  average  cost  for  an 
optimum  receiver.  This  result  is  a  generalization  of  a  similar  result  for  the  minimum  prob¬ 
ability  of  error  based  on  the  Kolmogorov  variational  distance.  The  result  obtained  here 
will  be  employed  in  the  next  few  chapters  for  decentralized  detection  systems. 
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CHAPTER  3 

BOUNDS  ON  THE  PERFORMANCE  OF  OPTIMUM 

RECEIVERS 


3.1  Introduction 

In  hypothesis  testing,  the  performance  of  optimum  receivers  is  usually  expressed  in 
terms  of  the  probability  of  error.  In  this  chapter,  we  concentrate  on  some  performance  as¬ 
pects  of  optimum  receivers.  We  consider  the  binary  hypothesis  testing  problem  in  which 
hypothesis  Hg  with  a  priori  probability  tcq  is  tested  against  hypothesis  Hj  with  a  priori 
probability  7Cj.  The  decision  is  made  based  on  a  random  observation  X  with  conditional 
probability  density  functions  fg(x)  and  fj(x)  when  Hg  and  Hj  are  true  respectively.  It  is 
well  known  that  the  optimum  receiver  which  minimizes  the  probability  of  error  is  the 
maximum  a  posteriori  probability  (MAP)  receiver  [3].  When  an  observation  x  is  re- 
ceived,the  MAP  receiver  computes  the  a  posteriori  probabilities  P(Hg!x)  and  PCHjIx)  and 
chooses  the  hypothesis  with  the  larger  a  posteriori  probability.  In  Chapter  2,  we  derived 
closed  form  expressions  for  the  minimum  probability  of  error  (MPOE)  and  the  general 
minimum  average  cost  (MAC)  of  the  optimum  receiver.  Analytic  evaluation  of  these  ex¬ 
pressions  is  very  difficult  in  most  cases  because  it  involves  the  evaluation  of  the  discontin¬ 
uous  function  min(.).  Instead  of  evaluating  the  exact  minimum  probability  of  error  and  the 
exact  minimum  average  cost,  tight  upper  and  /  or  lower  bounds  can  often  be  determined 
analytically  in  an  easier  fashion  making  it  possible  to  compare  the  performance  of  opti¬ 
mum  receivers  based  on  these  bounds. 

A  number  of  upper  and  lower  bounds  on  the  MPOE  of  optimum  receivers  have  been 
proposed  in  the  literature  [51-63].  Because  of  the  indirect  relationship  between  the  proba¬ 
bility  of  error  and  the  class  of  Ali-Silvey  distance  measures  (or  the  f-divergence)  [31],  a 
number  of  the  bounds  available  are  expressed  in  terms  of  these  distance  measures.  The 
idea  behind  this  relation  lies  in  the  result  known  as  the  Blackwell  theorem.This  theorem 
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states  the  following.  Let  it  be  the  set  of  all  permissible  pairs  of  the  prior  probabilities  7to 
and  7t]  in  a  binary  hypothesis  testing  problem.  Then  there  exists  a  subset  of  n  for  which  if 
the  distance  between  a  given  set  of  conditional  density  functions  is  larger  than  the  distance 
between  another  set  of  conditional  density  functions, then  the  probability  of  error  corre¬ 
sponding  to  the  first  set  is  less  than  the  probability  of  error  corresponding  to  the  second 
set.  Boekee  and  Van  der  Lubbe  [51]  provided  upper  bounds  on  the  MPOE  of  optimum  re¬ 
ceivers  by  considering  the  f-divergence  between  the  conditional  densities  under  the  two 
hypotheses.  They  have  shown  that  this  upper  bound  includes  many  well  known  bounds  in 
terms  of  other  distance  measures. 

As  discussed  before,  almost  all  of  the  literature  in  this  area  has  been  limited  to  find¬ 
ing  bounds  on  the  MPOE,  and  little  can  be  found  on  the  general  Bayesian  problem  in 
which  arbitrary  costs  are  assigned  to  each  course  of  action  in  the  decision  process.  In  this 
chapter  we  extend  some  of  the  known  bounds  on  the  MPOE  to  the  general  Bayesian  prob¬ 
lem.  In  addition,  we  obtain  a  new  upper  bound  on  the  MPOE  which  is  tighter  than  the  pre¬ 
viously  available  bounds.  We  also  obtain  a  tight  lower  bound  on  MPOE.  In  Section  3.2, 
we  derive  an  upper  bound  on  the  MAC  which  is  a  generalization  of  the  ChemofF  bound  on 
the  MPOE.  In  Section  3.3,  we  derive  simple  upper  and  lower  bounds  on  the  MAC  in  terms 
of  the  Bhattacharyya  coefficient.  In  Section  3.4,  we  introduce  our  new  upper  bound  on  the 
MPOE.  This  bound  is  shown  to  be  tighter  than  the  available  bounds  such  as  the  Bhatta¬ 
charyya  bound  and  the  equivocation  bound.  In  Section  3.5,  we  extend  this  bound  to  the  re¬ 
stricted  Bayesian  problem  in  which  Cqo  =  Cn  and  Cio  =  Cqi.  In  Section  3.6,  we  use  the 
upper  bound  derived  in  Section  3.4  to  obtain  a  new  tight  lower  bound  on  the  MPOE.  In 
Section  3.7,  we  present  a  numerical  example  where  we  compare  the  exact  minimum  prob¬ 
ability  of  error  with  the  new  upper  and  lower  bounds  on  the  minimum  probability  of  error. 
Section  3.8  contains  a  summary  of  the  results  obtained  in  this  chapter. 

3.2  Generalized  Chernoff  Bound 

In  this  section  we  derive  a  new  upper  bound  on  the  system  MAC  which  is  a  general¬ 
ization  of  the  Chernoff  bound  on  the  MPOE  available  in  the  literature  [52,53],  We  begin 
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with  (2.11)  and  (2.12)  which  are  repeated  here  for  convenience 


M*>  =  C00P(//0|i)+C01/'(Hl|Jc)  (3.1) 


P,(*)  =  C10P(W0|x)  +CnP  (//,|at)  (3.2) 


Using  Bayes  rule  we  can  express  the  a  posteriori  probabilities  PCHqIx)  and  P(Hjlx)  as 


Vo 

P('H°\X)  -  kJqM  +7C/J  (x) 


(3.3) 


Tt/l  (x) 

”  VoW  +K/i  U) 


(3.4) 


Using  (3.3)  and  (3.4),  the  expressions  for  Pq(x)  ^d  Pj(x)  in  (3.1)  and  (3.2)  become 


M*) 


COoVoW  +C017t1/,(x) 
Tt/O  (x)  +  Tt/j  (X) 


(3.5) 


PjU) 


cioVoW  +CllVt(x) 


(3.6) 


The  MAC  receiver  computes  the  quantities  PqW  and  (3](x)  and  makes  the  decision  ac¬ 
cording  to  decision  rule  (2.14).  The  conditional  cost  based  on  an  observation  x  is  given  by 


r(x)  =  m/n(P0(x),  (ij  (x) ) 


(3.7) 


For  any  two  positive  real  numbers  a  and  b,  we  have  the  following  inequality 
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(3.8) 


min(a,b)  <>asbX  s,  0<s<l 

Making  use  of  (3.8)  we  can  obtain  an  upper  bound  on  the  conditional  cost  r(x)  given  in 
(3.7)  as 

r( x)  <  ((30 U))i(p1  U))1-^, 0<^ <  1  (3.10) 

The  unconditional  cost,  R,  is  given  by 

R  =  Jmif!(P0(x),p1  (at))  +j ilfl(x))dx  (3.11) 

X 

Therefore,  an  upper  bound  on  R  is 

fl<J(P0(x))*(Pi  U))1",(jc(/0U)  +71/!  (jc))dx,0<5<  1.  (3.12) 

X 

Substituting  (3.5)  and  (3.6)  in  (3.12)  we  get 

/?  <  J(Coo7to/o  W  +  CoiTt/  (x))s (C107t/0  W  +cnn\f\ (x))l~sdx,0<s<  1. 

(3.13) 

The  upper  bound  in  (3.13)  is  true  for  any  value  of  s  in  the  range  0  <  s  <  1.  The  tightest 
bound  is  obtained  by  finding  the  particular  value  of  s  which  minimizes  the  right  hand  side 
of  (3.13).  Therefore,  the  tightest  upper  bound  of  this  form  on  MAC  is  given  as 

R~mios s<i  J <CooVoW  +C0iViW)J(CioVoW  +  cuK\fi  (*))l~sdx 

(3.14) 
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This  is  a  generalization  of  the  well  known  Chemoff  bound  on  the  minimum  probability  of 
error.  Using  the  special  cost  assignments  Cqo  =  C^  =  0  and  Cjq  =  =  1,  we  can  obtain 

it  from  (3.14)  as 


P(E)£min  J(j z/l(x))s(n0f0(x))1  5  dx  (3.15) 

~  X 


Let  s*  be  the  specific  value  of  s  that  achieves  the  minimum  in  (3.15).It  has  been  shown  in 
[53]  that  (3.15)  can  be  written  as 


P(E)  <7t(  TtJ-'V 


(3.16) 


where 


p*  =  min  f  (fj  (x)  )s  (f0(x))1~ *dx  (3.17) 

0<s<  1  { 


is  the  Chemoff  coefficient.  Equation  (3.16)  represents  the  usual  form  of  the  Chemoff 
bound  on  the  minimum  probability  of  error. 

3.3  Upper  and  Lower  Bounds  on  the  Minimum  Average  Cost  in  Terms  of 
the  Bhattacharyya  Coefficient 

The  evaluation  of  the  generalized  Chemoff  bound  in  (3.14)  is  a  difficult  task.  There¬ 
fore  it  is  worthwhile  to  derive  upper  and  lower  bounds  on  the  MAC  that  are  simpler  to 
evaluate.  Here  we  derive  bounds  in  terms  of  the  Bhattacharyya  coefficient.  This  coeffi¬ 
cient  is  obtained  from  (3.17)  by  using  the  specific  value  of  s  =  0.5,  i.e.,  the  Bhattacharyya 
coefficient  is  given  as 


PB  =  fjfi  U)/0  U)  dx 

X 


(3.18) 
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The  Bhattacharyya  coefficient  is  relatively  simple  to  evaluate  and  has  closed  form  expres¬ 
sions  for  many  commonly  used  distributions.  These  advantages  made  the  Bhattacharyya 
bound  on  the  minimum  probability  of  error  an  attractive  tool  in  the  design  of  communica¬ 
tion  systems.  The  most  common  applications  are  the  design  of  quantizers  for  hypothesis 
testing  [32]  and  signal  selection  [31].  The  main  disadvantage  of  the  Bhattacharyya  bound 
is  that  it  is  a  loose  bound.  In  this  section  we  derive  upper  and  lower  bounds  on  the  MAC 
based  on  the  Bhattacharyya  coefficient  which  are  generalizations  of  the  available  bounds 
on  the  MPOE.  These  generalized  bounds  are  based  upon  the  following  representation  of 
the  MAC  derived  in  Chapter  2 

R  =  Ro~  (*)  ~cofo(x)\dx  (3.19) 

X 

Here  we  make  the  usual  assumption  that  an  incorrect  decision  is  more  costly  than  a  correct 
decision,  Le.,  we  assume  that  C^q  >  Cqo  and  Cqi  >  Cu-  Equation  (3.19)  can  also  be  writ¬ 
ten  as 


JlCj/j  (x)  -  C(/0  (x)\dx  =  2*0-2/?  (3.20) 


Next,  we  obtain  the  upper  and  lower  bounds  on  the  MAC. 

3.3.1  Upper  Bound  on  the  MAC 

To  find  an  upper  bound  on  the  MAC  we  make  use  of  the  following  inequality  which 
is  true  for  any  two  positive  real  numbers  a  and  b 

\a-b\>  [Ja-  Jb]2  (3.21) 

With  the  aid  of  (3.21),the  left  hand  side  of  (3.20)  can  be  bounded  as 
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X 


X 


[  Vci/i  (*)  -  Jc of 0W]2 dx  (3.22) 

Expanding  the  right  hand  side  of  (3.22)  we  get 

\[JC1fl(x)-JC0f0(x)]2dx  =  J  (C/j  (x)  +  Cq/o  (x)  -  2^!  (x)/0  (x) )  rfx 

X  X 

(3.23) 

Integrating  term  by  term  on  the  right  hand  side  of  (3.23)  we  get 

^  [  Jc~^Tx)  ~  Jc^foix)  ]  dx  =  Cj  +  c0- 2^CjCqPb  (3.24) 

where  pB  is  the  Bhattacharyya  coefficient  defined  in  (3.18).  From  (3.20)  and  (3.22)  we  get 

2R0-2R>\[  JCJ^x)  -  VCo/oU)  ] 2  dx  (3.25) 

Substituting  (3.24)  into  (3.25)  we  obtain 


2R0-2R>C,+C0  2jC,C0pB  (3.26) 

Rearranging  the  terms  in  (3.26)  we  obtain  the  following  upper  bound  on  R 

R<R0-^(C^C0)+J^C0pb  (3.27) 

Using  the  definitions  of  Rq,  Cj  and  Cq  in  (2.2),  the  upper  bound  in  (3.27)  becomes 

R  <  CqqTZq  +  CjjTtj  +  (C01  -  Cn)  (Cjo ~  Qo)  Pb  (3.28) 
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which  is  the  desired  upper  bound. 


3.3.2  Lower  Bound  on  the  MAC 

In  this  subsection  we  derive  a  lower  bound  on  the  MAC  in  terms  of  the  Bhatta- 
charyya  coefficient  Here  we  make  use  of  the  Schwartz’s  inequality 


j /(o  g  (o  *<  rj  (fu) )  2dt\  ( g(t ) )  2dt 

t  Lz  t 


OJ 


(3.29) 


Squaring  both  sides  of  (3.29)  we  get 


[f 


(Og(t)dt 


2  r. 


j(f(t))2dti  r  j(g(o)2dt 


L  l 


J  Lr 


(3.30) 


For  any  two  real  numbers  a  and  b,  we  have  the  following  equality 


(a-b)  ( a  +  b )  =  a2-b 2 


(3.31) 


Let 


a  =  /Cj/j  (x) 


(3.32) 


b  -  JC^q  (jc) 


(3.33) 


Then  (3.31)  can  be  written  as 


(JCfx(x)  -JCofoix))  +JC0f0(x))  =  C/,U)  -Cq/ 0W 


(3.34) 


Taking  the  absolute  value  of  both  sides  of  (3.34)  and  integrating  with  respect  to  x  we  get 
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Jl  JcJt  U)  -  JC0f0(.x)\\JcjJ7)  +  JC,/0d)\dx  =  ||C,/,  ( x )  -  Cq/0  (x)  |£/jc 

X  X 

(3.35) 

Squaring  both  sides  of  (3.35)  we  get 


\\JCJX  U)  -  JCofoWWJCJ^x)  +  JCof0(x)\dx 


JjC^U)  -Cof0(x)\dx 


2 


(3.36; 


Let  us  make  the  following  substitutions  in  equation  (3.30) 


fit)  =  \JCxf:(t)  -JCof0(t) 


(3.37) 


git)  =  ^C^iO+JC^it) 


(3.38) 


Applying  Schwartz’s  inequality  (3.30)  to  the  left  hand  side  of  (3.36)  with  the  proper  sub¬ 
stitutions  from  (3.37)  and  (3.38)  we  get 


J|  Jc \A  ix)  -  Jcofo  ix) 1 1  Jc \f\  ix)  +  Jc of 0ix)\ dx 


2 


<IXI2 


(3.39) 


where 


/,  =  \\JcZW  -  Jc^wfdx 

X 


(3.40) 


h  =  j\JClf,(x)+JC0f0d)\2dx 

X 


(3.41) 
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Expanding  the  right  hand  side  of  (3.40)  and  (3.41)  and  integrating  we  get 


/j  =  C1  +  C0-2jclC0jJfl(x)f0(x)dx  (3.42) 


/2  =  Cl  +  C0  +  2jCiC0jJf1  (x)f0(x)dx  (3.43) 


It  is  easily  recognized  that  the  integral  on  the  right  hand  side  of  (3.42)  and  (3.43)  is  the 
Bhattacharyya  coefficient  pg.  But  the  left  hand  side  of  (3.39)  is  equal  to  the  right  hand  side 
of  (3.36).  Therefore,  we  have 


J|cifi(*)  -Cof0(x)\dx 


2 


< 


(Cj  +  C0  2pbJc^Cq)  (C j  +  C q  +  2pfl^CjC0) 


(3.44) 


Taking  the  square  root  of  both  sides  of  (3.44)  and  simplifying  we  get 


J|C,/,  (X)  -  C</0  (x)jdxs  J[Ct  +C0]2  -4CjCoPg  (3.45) 


Using  (3.20)  we  obtain 


2Ro-2R£j[C1  +  C0]2-4C1CopJ 


(3.46) 


Rearranging  the  terms  in  (3.46)  we  obtain  the  following  lower  bound  on  R 


«S*0-U[C1  +  C0]2-4C,C0p 


(3.47) 


Substituting  for  Rq,  Cq  and  Cj  from  (2.22),  we  have 
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2  (Coo  +  Cl0)  Tt0+2  (Cqi  +  ^n) 

“2^  (^01  “  ^11)  +  (C'io-  ^00)  —  (C01  ~  ^11)  (^10  ~  ^00)  P B 

(3.48) 

3.3.3  Minimum  Probability  of  Error  Bounds 

Here,  we  consider  the  minimum  probability  of  error  case,  i.e.,  when  Cqo  =  Cn  =  0, 
C10  =  Cqi  =  1  and  show  that  the  above  upper  and  lower  bounds  in  terms  of  the  Bhatta- 
charyya  coefficient  reduce  to  the  results  available  in  the  literature.  In  this  case,  the  upper 
bound  in  (3.28)  simplifies  to 


P(E)  <fio^pB 


(3.49) 


and  the  lower  bound  in  (3.48)  becomes 


(3.50) 


Equations  (3.49)  and  (3.50)  represent  the  upper  and  lower  bounds  on  the  probability  of  er¬ 
ror  which  are  well  known  [31].  These  bounds  are  convenient  from  a  computational  stand¬ 
point  but  are  rather  loose.  Therefore,  we  derive  tighter  upper  and  lower  bounds  on  the 
MPOE  in  the  rest  of  this  chapter. 

3.4  A  Tight  Upper  Bound  on  the  Probability  of  Error  for  Optimum 
Receivers 

For  the  MAP  receiver  discussed  in  Chapter  2,  the  conditional  probability  of  error 
based  on  an  observation  x  is  given  by 
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P(E\x)  =  min(P(//0|jc),/>(tfI|jc)) 


(3.51) 


For  notational  convenience  we  let 

\ 


p  =  />(//0|:r),0<p£  1 


(3.52) 


Then  we  have 


P{E\x)  =  min  (p,  1  -p)  =  g  (p) 


(3.53) 


The  function  g(p)  is  plotted  in  Fig.  3.1  as  a  function  of  p.  The  unconditional  probability  of 
error  is  the  expected  value  of  P(E  I  x)  with  respect  to  x,  i.e., 

P(E)  -  £x{F(£Ix)}  =  Ex  {g  (p)  } 


«  jg  (P)  (Vo  U)  +  ^  (3.54) 

X 


The  expression  for  the  minimum  probability  of  error  given  in  (3.54)  is  exact  but  is  compu¬ 
tationally  undesirable  in  many  applications  due  to  the  discontinuity  of  the  function  g(p)  at 
p  =  0.5.  Therefore,  we  attempt  to  find  a  computationally  desirable  function  of  p,  g*(p), 
which  when  substituted  for  g(p)  in  (3.54),  will  provide  a  close  approximation  or  a  tight 
bound  on  the  probability  of  error.  Let  B(g*)  denote  a  bound  on  the  probability  of  error 

B(g*)  =  J2*  ip)  (Vo  W  +  Vi  W )  (3-55) 

X 
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The  closer  the  function  g*(p)  is  to  g(p),  the  tighter  is  the  bound.  In  this  section,  we  find  an 
upper  bound  on  the  minimum  probability  of  error  based  on  the  function  g*(p)  which  satis¬ 
fies  the  following  conditions 

1)  g*(p)  >  g(p)  for  all  values  of  p  in  the  range  0  <  p  <  1  such  that  g*(p  =  0)  =  g*(p  =  1)  =  0. 
This  condition  is  consistent  with  the  requirement  that  g*(p)  be  an  upper  bound  function. 

2)  g*(p)  should  be  continuous  and  differentiable.  This  condition  is  desirable  to  avoid  the 
same  computational  difficulties  as  associated  with  the  exact  expression  based  on  g(p). 

3)  g*(p)  should  be  symmetrical  about  the  point  p  =  0.5.  This  condition  is  desirable  since 
g(p)  is  symmetrical  about  the  point  p  =  0.5. 

4)  g*(p  =  0.5)  =  0.5.  This  condition  is  needed  since  the  conditional  probability  of  error 
P(Elx)  cannot  be  larger  than  0.5  and  it  is  equal  to  0.5  only  when  p  =  0.5. 

5)  dg*/dp  (p  =  0)  >  1  and  dg*/dp  (p  =  1)  <  -1.  This  condition  ensures  that  the  function  se¬ 
lected  lies  above  the  function  g(p)  for  all  values  of  p  in  the  range  0  <  p  <  1. 

Now  we  demonstrate  that  several  upper  bounds  on  the  minimum  probability  of  error 
proposed  in  the  literature  can  be  interpreted  using  the  above  framework.  A  similar  idea 
has  been  pursued  by  Chen  [59].  First,  we  consider  the  upper  bound  based  on  the  Bhatta- 
charyya  coefficient  and  given  in  (3.49).  It  can  be  written  as 

P  (£)  £  J^i/j  (*)  V0  (*)  dx  (3.56) 


From  Bayes  rule  and  the  definition  of  p  in  (3.52)  we  have 

V0  (*)  =  P  (Vo  (*)  +  Vi  (*)  )  (3.57) 


V,  (*)  =  ( 1  -P)  (Vo  (*)  +  Vi  W  )  (3.58) 
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Substituting  (3.57)  and  (3.58)  into  (3.56)  we  get 


")  <jjp(\-p)  (Tt/o  m +  V  w  > dx 

X 


=  \gB  (P)  (Vo  (*)  +  Vi  (*) )  ^  (3.59) 

X 


where 


8b<J>)  *  Vp(l-P)  (3.60) 

Thus,  this  upper  bound  is  obtained  by  replacing  g(p)  by  gg  (p).  The  function  gg  (p)  will  be 
referred  to  as  the  Bhattacharyya  function  and  is  plotted  in  Fig.  3.1.  From  Fig.  3. 1 ,  we  see 
that  gg(p)  is  a  fairly  poor  approximation  to  the  function  g(p)  and,  therefore,  the  Bhatta¬ 
charyya  bound  is  a  loose  one.lt  can  be  easily  verified  that  the  function  gg  (p)  satisfies  all 
the  five  conditions  for  g*(p)  listed  above. 

Another  upper  bound  on  the  probability  of  error  that  is  available  in  the  literature  is  in 
terms  of  the  equivocation  (an  information  theoretic  measure).  This  upper  bound  is  given 
as 

P  (£)  <  —  Jo.5  [P  (H0 1  x)  log  P  (H0  \  x)  +  ( 1  -  P  (tf  oj  x) )  log  ( 1  -  P  (H0\  x)  ))f(x)  dx 

(3.61) 

In  this  case,  the  function  g(p)  is  replaced  by  the  equivocation  function  gg(p)  given  by 

gE  (p)  =  -0-5  [plogp  +  ( 1  -  p)  log  ( 1  -  p)  ]  (3.62) 
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Fig.  3.1  .  The  exact  minimum  probability  of  error,  the  new  upper  bound,  the  sinusoidal, 
and  the  Bhattacharyya  functions  plotted  as  a  function  of  p. 
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This  function  also  satisfies  the  five  conditions  listed  above.  It  has  been  shown  in  [55]  that 
the  equivocation  bound  given  by  (3.61)  gives  a  tighter  bound  on  the  probability  of  error 
than  the  Bhattacharyya  bound.  Devijver  [56]  introduced  a  new  bound  on  the  probability  of 
error  in  terms  of  what  is  called  the  Bayesian  distance.  This  distance  is  defined  as 

l 

By  =  £,{£  [P  (//,.)*)  ]2}  (3.63) 

i  =  o 


which  in  our  notation  reduces  to 

by  =  ExiP1+  O-P)2} 

=  Ex{l-2p(\-p)}  (3.64) 

The  probability  of  error  is  related  to  the  Bayesian  distance  through  the  relation 

P(E)  <1  -By 

<  Ex{2p(l-p)}  (3.65) 

In  this  case,  the  function  g(p)  is  replaced  by  the  function 

gY(p)  =  2p  (1  -p)  (3.66) 


This  function  will  be  called  the  Bayesian  bound  function.  It  has  been  verified  in  [56]  that 
the  Bayesian  bound  function  yields  a  tighter  bound  on  the  probability  of  error  than  the 
equivocation  function.  As  shown  in  Fig.  3.1,  the  function  gy  (p)  approximates  g(p)  much 
more  closely  than  gg  (p).  We  have  not  shown  gg  (p)  in  Fig.  3.1  but  it  falls  between  the 
functions  gg  (p)  and  gy  (p).  Next,  we  develop  a  new  function  that  satisfies  the  desirable 
conditions  indicated  earlier  and  approximates  g(p)  even  more  closely.  The  resulting  func- 
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tion  will  yield  a  tighter  upper  bound  on  the  minimum  probability  of  error. 


Consider  the  sinusoidal  function 

gs(p)  =  0.5  sin7tp  (3.67) 


This  function  satisfies  the  desired  properties  of  the  approximating  function  as  described 
earlier.  Furthermore,  for  all  values  of  p  in  the  range  0  <  p  <  1,  we  have  gs  (p)  <  gy  (p),  i.e., 

0.5 sirnzp  <2p  (l  - p)  (3.68) 


The  result  in  (3.68)  indicates  that  the  sinusoidal  function  gs(p)  will  lead  to  a  tighter  upper 
bound  than  the  Bayesian  bound  since  it  is  closer  to  the  minimum  probability  of  error  func¬ 
tion  g(p).  Differentiating  the  sinusoidal  function  in  (3.67)  and  evaluating  the  derivative  at 
p  =  0,  we  see  that  the  slope  at  p  =  0  is  1.57.  This  value  of  the  slope  is  much  larger  than  the 
slope  of  g(p)  at  p  =  0  which  is  equal  to  1.  We  can  tighten  the  bound  obtained  from  (3.67) 
by  weighting  the  sinusoidal  function  by  a  Gaussian  function,  and  we  assume  that  the  ap¬ 
proximation  function  is 

gN(p)  =  0.5  (sirnzp)  exp  [-a (p-0.5)2]  (3.69) 


The  Gaussian  function  is  continuous,  differentiable,  and  symmetrical  about  the  point  p  = 
0.5  as  required  for  the  approximating  function.  The  reason  for  including  the  Gaussian 
function  in  g^  (p)  is  that  it  exhibits  exponential  decay.  This  property  makes  it  possible  to 
better  approximate  the  exact  function  g(p)  with  the  sinusoidal  and  the  Gaussian  functions 
combined  than  with  the  sinusoidal  function  alone.  The  parameter  a  determines  the  rate  of 
decay  of  the  Gaussian  function.  This  parameter  is  to  be  chosen  so  that  the  derivative  of 
gjsj(p)  at  p  =  0  is  equal  to  1.  We  recall  that  the  derivative  of  the  function  g(p)  at  p  =  0  is 
equal  to  1.  Therefore,  by  letting  the  derivative  of  gjsr(p)  at  p  =  0  to  be  equal  to  1,  we  are 
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forcing  both  functions  gN(p)  and  g(p) to  have  the  same  slope  at  the  points  p  =  0  and  p  =  1. 
Furthermore,  the  values  of  the  two  functions  at  the  points  p  =  0  and  p  =  1  are  equal  to  zero. 
Therefore,  by  having  equal  values  of  the  two  functions  as  well  as  equal  slopes  at  the  points 
p  =  0  and  p  =  1,  we  can  obtain  the  best  approximation  possible  for  the  function  considered. 
Differentiating  (3.69)  with  respect  to  p,  setting  the  derivative  at  zero  equal  to  one,  and 
solving  for  a  we  get  a  =  1.8063.  So  our  new  approximating  function  to  obtain  the  upper 
bound  is 


gN(p )  =  0.5  (sinjip)  exp  [-1.8063  (p- 0.5) 2]  (3.70) 


This  function  is  plotted  in  Fig.  3.1.  In  Fig.  3.2,  we  plot  the  functions  g(p)  and  gs(p)  versus 
p.  We  also  plot  the  function  gN(p)  to  show  the  improvement  gained  by  weighting  the  sinu¬ 
soidal  function  gs(p)  by  the  Gaussian  function.  By  comparing  the  various  approximation 
functions  in  Figures  3. 1  and  3.2,  we  see  that  the  new  function  given  in  (3.70)  approxi¬ 
mates  the  exact  function  g(p)  quite  closely,  and  that  it  is  significantly  better  than  the  Baye¬ 
sian  bound  function.  The  probability  of  error  is  now  upper  bounded  by 

P  (£)  <  J  (0.5  (sinrcp)  exp  [-1.8063  (p  —  0.5) 2] )  (kJo  (x)  +  7t,/j  (x) )  dx 

X  (3.71) 

In  this  section  we  have  presented  a  new  upper  bound  on  the  minimum  probability  of 
error  which  is  shown  to  be  tighter  than  the  presently  available  bounds  such  as  the  Bhatta- 
charyya,  the  equivocation,  and  the  Bayesian  bounds.  Some  applications  of  the  new  bound 
in  the  design  of  decentralized  detection  systems  will  be  considered  in  the  next  chapter. 

3.5  A  Tight  Upper  Bound  on  the  Minimum  Average  Cost 

In  the  previous  section,  we  derived  a  tight  upper  bound  on  the  probability  of  error  for 
optimum  receivers.  This  is  a  special  case  of  the  general  Bayesian  problem  where  the  costs 
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Fig.  3.2  .  Sinusoidal  and  the  Gaussian  weighted  sinusoidal  functions. 
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are  assigned  as  Cqq  =  Cjj  =  0  and  Cjo  =  Cqi  =  1.  In  this  section  we  extend  the  bound  to  the 
more  general  Bayesian  hypothesis  testing  problem  with  symmetrical  costs,  i.e.,  with  costs 
satisfying  Coo  =  C11  c10  =  Q)l-  Expressing  P0(x)  in  (3.1)  and  Pi(x)  in  (3.2)  in  terms 

of  P(Hq  I  x),we  get 

P0U)  =  CnPiHo^+Cnd-PiH^x))  (3.72) 


PjU)  =  Cl0P(H0\x)  +Cn(l-P(H0\x)) 


Rearranging  terms  in  (3.72)  and  (3.73)  we  get 


P0U)-C01  ~  (Coo  C01) P (Hq\x) 


(3.73) 


(3.74) 


M*)  -Cu  =  (Cl0-Cn)P(H0\x)  (3.75) 


Dividing  (3.74)  by  (3.75)  we  get 

Po(*)-Coi  __  Cqq  -  C01 

Pj  (*)  ~  C\i  Cjq-  Cn 

From  (3.76)  we  express  Pq(x)  in  terms  of  Pj(x)  as 

P0U)  =  -Apj  (x)  +B 


(3.76) 


(3.77) 


where 
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(3.78) 


D  C01Cio  CoqCh 

B - r  t r - 

^10  L'll 

Using  (3.77),  the  conditional  cost  in  (3.7)  becomes 

r(x)  =  min  (-A^  (x)  +  5,  p1  (x) ) 


(3.79) 


(3.80) 


As  P(Ho  I  x)  varies  over  the  range  0  <  P(HqIx)  <  1,  Pq(x)  in  (3.72)  varies  over  Cq]  <  Pq(x) 
<  Coo  and  Pi(x)  in  (3.73)  varies  over  ^  Pi(x)  ^  .  For  an  arbitrary  cost  assignment 

the  functions  Pi(x)  and  -AP^xJ+B  as  a  function  of  p](x)  are  sketched  in  Fig.  3.3  .  Also 
shown  in  this  figure  is  the  minimum  of  the  two  functions  which  represents  r(x).  The  lack 
of  symmetry  of  the  conditional  cost  function  makes  it  difficult  to  extend  the  upper  bound 
obtained  in  Section  3.4  to  the  general  Bayesian  case.  Therefore,  we  consider  the  restricted 
problem  with  symmetrical  costs  in  which  Cqq  =  Cn  and  Cjq  =  Cqi-  In  this  case  A  =  1  and 
B  =  Coi+Cn-  The  conditional  cost  in  (3.80)  becomes 

r(x)  =min(-fi](x)+C01  +  Cu,fi1(x))  (3.81) 


The  functions  Pj(x)  and  -P1(x)+Coi+C11  are  sketched  in  Fig.  3.4  alongwith  the  minimum 
of  the  two  functions.  As  can  be  seen  from  Fig.  3.4,  the  conditional  cost  r(x)  is  a  scaled  and 
translated  version  of  the  minimum  probability  of  error  function  g(p).  Figure  3.5-a  shows 
the  function  r(x)  -  Cjj  resulting  from  the  translation  of  the  function  r(x)  along  the  vertical 
axis.  Figure  3.5-b  shows  the  function  resulting  from  translating  the  function  r(x)  -  Cu 
along  the  horizontal  axis  to  the  origin  by  C\  j.  The  function  g/  (Pj)  sketched  in  Fig.  3.5-b  is 
represented  in  terms  of  g(Pi)  as 
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ctions  Pi(x)  and 


Pl(x)+Cio+Cn  for  arbitrary  cost  assignment. 
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Fig.  3.4.  The  functions  f}j(x)  and  -  f}j(x)+Cio+Cu  for  the  case  of  symmetrical  costs. 
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f  Pi  ^ 

5/(^)  “\(Ci°"Cn)g[c10-Clly 

\ 


(3.82) 


while  the  function  sketched  in  Fig.  3.5-h  is  represented  as 


\ 


r(x)  -Cu  =  (Cj0-Cn)* 


f  Pi  C.n  N 

^lo-Cn  j 


(3.83) 


Using  (3.83),  the  function  sketched  in  Fig.  3.4, can  be  expressed  as 


r  (x)  —  C  jj  +  (Cjo  Cj  j)g 


f  Pi  ^li  N 

^cio_cn  > 


(3.84) 


Therefore,  using  (3.70),  we  can  upper  bound  the  conditional  cost  r(x)  using  the  function 


f  P,  Cjj 
S*(P,)  -  Ci,f(C10-C,,)gv  1  _ 

Vcio  cn 


(3.85) 


which  can  explicitly  be  expressed  as 


Sr  (Pi)  ~  Cn  +  0.5  (C10  C^j)  sin 


.  MPj-Cn) 


Cio  ; 


exp 


-1.8063 


rP,-c 


li 


kCio 


-  0.5  J 
(3.86) 


But  Pi  is  a  function  of  the  random  variable  X  as  se^n  from  (3.73).  We  emphasize  this  by 
expressing  Pi  as  Pi(x).  Therefore,  the  MAC  is  upper  bounded  by 

i?^£z{g^(p1(x))}  =  JijfCPjCx))  (Vo  (*)  +n/i(x))dx  (3.87) 


Next,  we  derive  a  lower  bound  on  the  minimum  probability  of  error. 
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3.6  A  Tight  Lower  Bound  on  the  Probability  of  Error  for  Optimum  Re¬ 
ceivers 

In  Section  3.5,  we  obtained  a  tight  upper  bound  on  the  probability  of  error  for  opti¬ 
mum  receivers.  We  listed  several  desirable  conditions  which  an  approximating  function 
g*(p)  should  satisfy  in  order  to  find  a  suitable  upper  bound.  Now  we  present  a  similar  set 
of  conditions  that  an  approximating  function  g**(p)  should  satisfy  in  order  to  find  a  lower 
bound.  These  conditions  are 

1)  g**(p)  should  be  continuous  and  differentiable. 

2)  g**(p)  should  be  symmetrical  about  the  point  p  =  0.5.  This  condition  is  desirable  since 
g(p)  is  symmetrical  about  the  point  p  =  0.5. 

3)  g**(p  =  0)  =  g**(p  =  1)  =  0. 

4)  dg**/dp  (p  =  0)  <  1  and  dg**/dp  (p  =  1)  >  -1. 

5)  g**(p)  ^  g  (p  )  for  all  values  of  p  in  the  range  0  <  p  <  1. 

A  lower  bound  on  the  minimum  probability  of  error  can  be  derived  in  terms  of  the 
new  upper  bound  function  g^(p)  discussed  in  Section  3.5.  The  first,  second,  and  third  de¬ 
sirable  conditions  for  the  lower  bound  given  above  are  satisfied  by  the  function  gN(p).  The 
fourth  condition  in  conjunction  with  the  fifth  are  needed  to  guarantee  that  the  lower  bound 
function  is  below  the  function  g(p)  for  all  values  of  p  in  the  range  0  <  p  <  1.  Now  we  pro¬ 
pose  a  lower  bound  function  of  the  form 

gL(p)  =  ex0.5  (sin7tp) exp  [-1.8063  (p-0.5)2]  (3.88) 


The  scaling  factor  e  is  a  constant  yet  to  be  determined  whose  value  is  less  than  unity.  With 
the  value  of  £  <  1,  the  fourth  condition  is  satisfied.  The  fifth  condition  is  satisfied  when 


e0.5  ( sin  7t/7)e.rpl>l.  8063  (/?- 0.5  )2]  <p,0  <p  <0.5  (3.89) 
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Our  objective  is  to  determine  those  values  of  e  for  which  (3.89)  is  satisfied.  A  detailed  ex¬ 
amination  of  (3.89)  shows  that  it  is  satisfied  for  the  following  values  of  e 

0  <  e  <  0.79  (3.90) 

With  the  value  of  e  =  0.79,  (3.89)  is  achieved  with  equality  for  a  specific  value  of  p.  The 
point  p  at  which  this  happens  is  p  =  0.275.  The  tightest  lower  bound  is  achieved  by  using 
£  =  0.79.  Using  this  value  of  £,  we  see  that  the  lower  bound  on  the  minimum  probability  of 
error  is  given  by 

P  ( E )  >  |  (0.395  (sinttp)  exp  [-1.8063  (p  -  0.5 ) 2] )  ( x )  +  Ttj/j  (x) )  dx 

x  (3.91) 

Comparing  (3.91)  and  (3.71),  we  see  that  the  relationship  between  the  upper  bound  and 
the  lower  bound  can  be  stated  as 

Lower  bound  =  0.79  x  Upper  bound  (3.92) 

In  Fig.  3.6  we  plot  the  functions  gL(p).  gisKP)’  a^d  g(p)  as  a  function  of  p.  As  seen  from 
this  figure,  the  lower  bound  function  closely  approximates  the  function  g(p)  for  most  of 
the  values  of  p  except  around  the  point  p  =  0.5.  The  tightness  of  the  bounds  can  also  be 
expressed  as  the  ratio  of  the  upper  bound  to  the  lower  bound.  This  ratio  should  be  as  close 
to  unity  as  possible.  For  the  new  bounds  derived  in  this  chapter  this  ratio  is  equal  to  1.265 
indicating  that  both  the  upper  and  the  lower  bounds  are  tight  bounds. 

The  same  argument  as  above  applies  to  the  case  of  symmetrical  cost  assignment,  i.e., 
a  lower  bound  function  on  the  cost  can  be  defined  in  terms  of  the  upper  bound  function 
gR(Pl)  as 


*«.(  Pi>  =  0.79  gR(^) 


(3.93) 
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Fig.  3.6.  The  exact  mimimum  probability  of  error,  the  new  upper  bound,  and  the  new 
lower  bound  functions  plotted  as  a  function  of  p. 
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The  MAC  can  therefore  be  lower  bounded  by 


=  0.79js*  (p^*))  (VoW  +  7t/i  M)dx 

x  (3.94) 

3.7  Example 

In  this  section  we  present  a  numerical  example  where  we  compare  the  exact  mini¬ 
mum  probability  of  error  in  a  hypothesis  testing  problem  with  the  new  upper  and  lower 
bounds  on  the  minimum  probability  of  error  derived  in  this  chapter.  For  the  sake  of  com¬ 
parison  we  also  consider  the  Bhattacharyya  upper  and  lower  bounds. 

Consider  the  hypothesis  testing  problem  in  which  the  observation  X  under  Hg  is  a 
Gaussian  random  variable  with  mean  mg  and  variance  Og  and  when  Hj  is  true  the  obser- 
vation  X  is  a  Gaussian  random  variable  with  mean  mj  and  variance  Oj.  The  hypothesis 
testing  problem  can  be  represented  as 

Hg  :  Xj  -  N(mg,  o02  ) 

Hj  :  Xj  ~  N(mlt  ) 

We  assume  that  rig  =  rcj  =0.5.  The  probability  of  error  is  minimized  using  the  likelihood 
ratio  test  given  by  (2.2) 

Hi 

fi(x)  > 

-  1  (3.95) 

M*)  < 

Ho 

where 

,  i  =  0,  1  (3.96) 

For  the  Gaussian  density  functions  given  by  (3.96),  the  Bhattacharyya  coefficient  is  given 
by  [31] 
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PB  =  exp 


[  (mt  -m0)2  1  f^  +  gjh 

[  4(Oj+aJ)  2  v2oiao> 


(3.97) 


We  consider  two  cases.  In  the  first  case  we  assume  that  Oq2=  Oj2  =  1,  =  0  and  mj  is  a 

variable.  In  Fig.  3.7  we  plot  the  exact  minimum  probability  of  error  as  a  function  of  mi 
when  decision  rule  (3.95)  is  employed  alongwith  the  new  upper  and  lower  bounds  on  the 
minimum  probability  of  error  determined  from  (3.71)  and  (3.91).  We  also  plot  the  Bhatta- 
charyya  upper  and  lower  bounds  determined  from  (3.49)  and  (3.50).  In  the  second  case, 
we  assume  that  mo  =  0,  =  2  ,  Cq2  =1  and  c^2  is  a  variable.  The  exact  minimum  proba¬ 

bility  of  error,  the  new  upper  and  lower  bounds,  and  the  Bhattacharyya  upper  and  lower 
bound  as  a  function  of  aj2  are  plotted  in  Fig.  3.8.  Figures  3.7  and  3.8  exhibits  the  tight¬ 
ness  of  the  new  upper  and  lower  bounds. 


3.8  Summary 

In  this  chapter  we  derived  upper  and  lower  bounds  on  the  minimum  probability  of 
error  and  the  minimum  average  cost  for  optimum  receivers.  Some  of  the  bounds  on  the 
MAC  are  generalizations  of  the  known  bounds  on  the  minimum  probability  of  error  in 
terms  of  the  Bhattacharyya  bound  and  the  Chemoff  bound.  Furthermore,  we  derived  a  new 
upper  bound  on  the  probability  of  error  which  is  tighter  than  the  previously  available 
bounds.  This  bound  was  generalized  to  the  Bayesian  problem  with  symmetrical  costs. 
Tight  Lower  bounds  on  the  probability  of  error  and  the  MAC  were  also  derived  in  terms  of 
the  new  upper  bound.  These  bounds  are  used  for  the  performance  characterization  of  opti¬ 
mum  receivers.  In  addition,  these  bounds  can  be  employed  for  the  design  of  quantizers  in 
signal  detection  systems,  for  the  design  of  decentralized  detection  systems,  for  signal  de¬ 
sign  and  other  related  problems  in  communication  systems. 
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Fig.  3.7.  Comparison  of  the  new  bounds,  the  Bhattacharyya  bounds,  and  the  exact 
minimum  probability  of  error  when  mi  is  varied. 
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Fig.  3.8.  Comparison  of  the  new  bounds,  the  Bhattacharyya  bounds,  and  the  exact 
minimum  probability  of  error  when  O]2  is  varied. 


CHAPTER  4 


PERFORMANCE  ANALYSIS  AND  DESIGN  OF  THE 
BAYESIAN  PARALLEL  FUSION  SYSTEM 

4.1  Introduction 

As  discussed  in  Chapter  1,  in  a  decentralized  detection  system  with  data  fusion,  a 
group  of  local  detectors  process  the  observations  they  receive  regarding  the  status  of  a  cer¬ 
tain  phenomenon,  and  transmit  their  decisions  to  a  fusion  center  where  the  final  decision  is 
made.  The  design  of  this  system  involves  specifying  both  the  local  decision  rules  and  the 
fusion  rule.  Several  authors  have  dealt  with  the  design  of  such  a  system  using  different  cri¬ 
teria.  Tenney  and  Sandell  [7]  treated  the  decentralized  detection  problem  with  no  data  fu¬ 
sion  from  a  Bayesian  point  of  view.  Costs  were  assigned  to  reflect  the  various  courses  of 
actions  of  each  local  detector.  The  local  decision  rules  were  chosen  such  that  the  average 
cost  is  minimized.  Assuming  known  sensor  thresholds.  Chair  and  Varshney  [9]  developed 
a  minimum  average  cost  algorithm  for  combining  the  sensor  decisions  at  the  fusion  center. 
Optimization  of  the  entire  system  was  considered  by  Hoballah  and  Varshney  [10]  where 
they  obtained  a  person-by-person  optimal  solution  to  the  problem.  The  optimization  of  the 
entire  system  was  also  considered  by  Reibman  and  Nolte  [12]  where  an  exhaustive  search 
has  to  be  done  over  all  the  possible  fusion  rules  in  order  to  determine  the  overall  minimum 
cost  solution.  Chair  and  Varshney  [14]  considered  the  problem  of  distributed  Bayesian  hy¬ 
pothesis  testing  with  distributed  data  fusion  in  which  data  fusion  is  performed  at  each  site. 

Distance  measures  from  the  class  of  Ali-Silvey  distance  measures  have  recently 
been  used  for  the  design  of  decentralized  detection  systems  [35,38].  In  fact,  this  class  has 
received  an  increasing  interest  in  the  design  of  quantizers  for  hypothesis  testing  because  of 
its  strong  link  to  the  probability  of  enor  (POE).  Let  n  =  {ttg,  7Cj }  be  the  set  of  all  permissi¬ 
ble  pairs  of  the  prior  probabilities  in  a  binary  hypothesis  testing  problem.  Then,there  exists 
a  subset  of  n  for  which  if  the  distance  between  a  given  set  of  conditional  densities  is  larger 
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than  the  distance  between  another  set  of  conditional  densities,  then  the  POE  corresponding 
to  the  first  set  is  less  than  the  POE  corresponding  to  the  second  set.  This  result  is  known  as 
the  Blackwell  theorem  [31].  Poor  and  Thomas  [32]  applied  the  general  class  of  Ali-Silvey 
distance  measures  to  the  design  of  a  generalized  quantizer  for  binary  decision  systems. 
Poor  [33]  also  used  these  measures  as  criteria  for  analyzing  the  effects  of  fine  data  quanti¬ 
zation  on  inferential  procedures  and  for  designing  quantizers  to  minimize  these  deterimen- 
tal  effects.  A  number  of  authors  have  applied  members  of  the. class  of  Ali-Silvey  distance 
measures  into  the  design  of  decentralized  detection  systems.  For  example,  Longo,  Looka- 
baugh,  and  Gray  [35]  have  employed  the  Bhattacharyya  distance,  whereas  Lee  and  Chao 
[36]  have  used  the  J-divergence  to  subpartition  the  decision  space  when  a  quality  bit  is  to 
be  transmitted  along  with  the  decision  to  the  fusion  center.  Kazakos  [57]  has  employed 
these  distance  measures  to  obtain  bounds  on  the  performance  of  distributed  detection  sys¬ 
tems. 

In  this  chapter  we  employ  the  minimum  average  cost  (MAC)  as  the  system  perfor¬ 
mance  measure  for  the  parallel  fusion  system  shown  in  Fig.  1.1.  Here  our  objectives  are 
threefold.  First,  we  shall  examine  the  improvement  in  the  system  performance  as  a  func¬ 
tion  of  the  number  of  sensors.  To  achieve  this  objective,  we  need  an  explicit  relationship 
between  the  minimum  average  cost  and  the  number  of  sensors.  Second,  we  shall  obtain 
optimum  local  thresholds  which  minimize  the  global  average  cost  for  a  given  number  of 
sensors.  Third,  as  an  example,  we  consider  the  design  of  the  minimum  probability  of  error 
(MPOE)  detection  system  where  we  compare  the  performance  of  the  optimum  system  t 
several  suboptimum  systems. 

The  chapter  is  organized  in  six  sections.  Section  4.2  contains  the  system  description, 
terminology,  and  notation.  In  Section  4.3,  we  examine  a  two-sensor  system  and  derive  a 
condition  under  which  the  performance  of  this  system  is  identical  to  that  of  a  single  sensor. 
We  also  derive  relationships  for  the  minimum  cost  of  an  n-sensor  system  in  terms  of  the 
sensor  decisions  for  the  hard  and  the  soft  decision  cases.  In  Section  4.4,  we  make  use  of 
the  results  derived  .n  Section  4.3  to  design  a  system  with  minimum  average  cost  In  Sec¬ 
tion  4.5,  we  consider  the  design  of  the  global  minimum  probability  of  error  systems.  In  ad- 
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dition,  we  discuss  a  suboptimum  system  design  in  which  all  the  system  components  are 
MAP  receivers.  Then,  we  discuss  two  suboptimum  systems  designed  based  on  the  dis¬ 
crimination  and  the  Bhattacharyya  distance  measures.  Next,  we  employ  the  new  upper 
bound  on  the  minimum  probability  of  error  derived  in  Chapter  3  to  design  a  nearly  opti¬ 
mum  decentralized  detection  system.  Finally,  we  present  numerical  examples  in  which  we 
compare  the  performance  of  the  various  suboptimum  systems  to  that  of  the  optimum  sys¬ 
tem.  We  also  compare  the  performance  of  the  nearly  optimum  system  to  that  of  the  opti¬ 
mum  system.  Section  4.6  contains  a  summary  of  the  work  reported  in  this  chapter. 

4.2  Preliminaries 

Let  us  consider  the  system  shown  in  Figure  1.1  consisting  of  n  local  detectors  and  a 
fusion  center.  Here,  we  consider  the  simple  binary  hypothesis  testing  problem.  The  hy¬ 
pothesis  Hq  with  a  priori  probability  tiq  is  tested  against  the  alternative  hypothesis  Hj  with 
a  priori  probability  tcj.  The  system  receives  n  observations  X],  X2,...,  Xn  where  X,  denotes 
the  observation  received  by  local  detector  LDj.  We  limit  our  treatment  to  the  case  when  Xj 
is  a  scalar  observation.  The  generalization  to  the  vector  case  is  straightforward.  We  as¬ 
sume  that  these  observations  are  independent  and  identically  distributed  random  variables 
with  conditional  probability  density  functions  pj(xj) ;  j  =  0,  1,  i  =  1, ....  n.  The  local  detec¬ 
tors  LDj, ...,  LDn  are  linked  to  the  global  decision  maker  (also  known  as  the  fusion  center) 
through  bandlimited  channels.  Due  to  this  constraint  on  the  channel  capacities,  the  local 
detectors  transmit  a  compressed  version  of  the  observed  data  to  the  fusion  center.  The 
compressed  data  can  be  in  the  form  of  a  hard  decision  or  a  soft  decision.  In  the  first  case, 
the  local  decision  Zj  takes  on  one  of  two  possible  values  depending  upon  the  local  decision 
of  local  detector  LDj.  In  the  second  case,  the  observation  space  of  X,  is  partitioned  into  M 
nonoverlapping  regions.  The  transmitted  local  decision  zj,  correspondingly,  takes  on  one 
of  M  possible  values.  In  both  cases  the  fusion  center  is  responsible  for  making  the  final  de¬ 
cision. 

For  M  >  2,  let  Tjq  <  Tjj  <...  <  T^  be  the  local  thresholds  of  local  detector  LDj  with 
Tjo  =  ' 00  and  TjM  =  00.  Also,  let  {aj], ....  ajjvi)  be  the  set  of  values  that  Zj  may  assume. 
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These  outcomes  occur  with  the  following  probabilities 


T* 

Pikj  =  P  (?r  <*,•*]  Hj)  =  j  Pj  (*,)  dxi>  1  £  i  ^  #i,  1  ^  k  <M,j  =  0, 1. 

(4.1) 


We  assume  that  the  decision  rules  at  the  local  detectors  are  identical.  This  means  that  T^  = 
Tfc  and  P&j  =  Pkj  for  1  <  i  <  n  It  should  be  pointed  out  that  the  theory  presented  here  can 
be  easily  generalized  to  the  case  where  local  decision  rules  are  not  identical.  For  simplicity 
of  presentation,  we  limit  our  discussion  in  this  chapter  to  the  identical  local  detector  case. 
It  is  worth  mentioning  that  it  has  been  observed  in  [24]  that  for  decentralized  Bayesian  de¬ 
tection,  identical  decision  rule  assumption  often  results  in  little  or  no  loss  of  optimality.  In 
the  next  chapter  we  deal  with  the  situation  where  the  local  decision  zj  is  either  0  or  1  and 
where  the  observations  are  not  identical.  We  also  assume  that  the  decision  made  by  local 
detector  LDj  is  independent  of  the  decisions  made  by  the  other  local  detectors.  Based  on 
these  assumptions,  the  decisions  zj, ...,  form  a  sequence  of  i.i.d  generalized  Bernoulli 
trials  with  parameters  (Pjj,  P2j, ....  PMj)  when  Hj  is  true.  These  decisions  are  sent  over 
bandlimited  channels  to  the  fusion  center.  The  fusion  center  which  is  the  global  decision 
maker  bases  its  decision  on  the  decision  vector  U=[zj, ...  zj.  To  optimally  partition  the 
observation  space  formed  by  the  discrete  random  vector  U  into  the  decision  regions  corre¬ 
sponding  to  Hq  and  H1?  the  fusion  center  performs  a  likelihood  ratio  test  Our  treatment  in 
this  chapter  is  based  on  the  assumption  that  the  fusion  center  is  a  minimum  average  cost 
(MAC)  receiver.  If  C^j ;  k,  j  =  0, 1,  denotes  the  overall  cost  of  deciding  when  Hj  is  true 
then  the  global  decision  rule  corresponding  to  this  choice  of  the  fusion  center  is 


UQ= 


if  AQJ)  >T| 
otherwise 


(4.2) 


where 


P(U\HX)  _  P(U=  zv.  zn\Hx ) 
"  P(U\H0)  '  P(U=  zv..znH0) 
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Vcio-Coo) 

”  '  *,(Coi-Cn) 

and  uo  =  k  means  that  the  fusion  center  decides  in  favor  of  Hk.  Making  use  of  the  indepen¬ 
dence  assumption  between  the  decisions  Zj,  A(U)  can  be  expressed  as 

n 

np<2,iw.> 

AW)  =  ^ -  (4.3) 

np<2,i«o) 

i=  1 

Using  (2.29),  the  MAC  of  the  system  can  be  expressed  as 


R  =  *0-0.5£ 

u 


c.r^u^-corp^o) 


i  =  i 


i=  1 


(4.4) 


where  the  summation  is  taken  over  all  values  assumed  by  the  vector  U=[zj  Z2, ...  zj  with 
each  element  Zj  taking  one  of  the  M  possible  values.  For  the  interesting  case  of  M  =  2, 
hard  decisions  are  made  at  the  local  detectors  based  on  the  following  decision  rule 

{1  ifpxfxJ/ptfxJZx 

i=l,2 . n  (4.5) 

0  otherwise 

where  zj  =  1  means  that  Hj  is  declared  true  and  z,  =  0  means  that  Hq  is  declared  true  by  lo¬ 
cal  detector  LDj.  It  should  be  pointed  out  that  the  decision  rule  (4.5)  is  a  threshold  test  due 
to  the  independence  assumption  made  earlier.  This  decision  rule  thus  characterizes  each 
local  detector  in  terms  of  its  probabilities  of  detection  PD  and  of  false  alarm  PF  The  deci¬ 
sion  Zj  has  the  following  conditional  density  functions  under  the  two  hypotheses 

/o<*i>  =  Ppd (4.6) 

/o<*i>  =  (47) 
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and  the  global  likelihood  function  (4.3)  becomes 

n/i  <z«) 

A(U)  =  ^ -  (4.8) 

n/o(*i) 

i = i 

Based  on  the  likelihood  ratio  test  (4.2)  and  using  (4.3)  and  (4.8),  we  derive  in  Appendix  A 
the  algorithm  based  on  which  the  fusion  center  combines  the  decisions  received  from  the 
local  detectors  in  order  to  make  the  global  decision. 

At  this  point  we  need  to  make  a  few  remarks  regarding  (4.2)  and  (4.4). 

1)  The  specific  values  of  U  that  satisfy  equality  in  the  likelihood  ratio  test  (4.2)  can  be  as¬ 
signed  to  either  one  of  the  decision  regions  corresponding  to  Hq  and  Hj  without  affecting 
the  MAC,i.e.,  no  randomization  is  necessary. 

2)  Of  all  the  possible  fusion  rules,  (4.2)  specifies  the  fusion  rule  that  can  achieve  the  small¬ 

est  MAC  for  given  local  decision  rules.  Equation  (4.4)  specifies  the  cost  corresponding  to 
this  (best)  fusion  rule.  It  should  be  pointed  out  that  an  explicit  knowledge  of  the  fusion 
rule  is  not  required  to  determine  the  MAC  given  in  (4.4).  The  following  example  which 
deals  with  the  binary  decision  case  illustrates  this  point.  ^ 

Example  4.1 

Consider  the  decentralized  detection  system  of  Fig.  1.1  with  three  local  detectors. 
Let  us  consider  the  hypothesis  testing  problem  in  which  the  observations  Xj  are  normally 
distributed  with  unit  variance,  zero  mean  when  Hq  is  true  and  unit  mean  when  Hj  is  true, 
i.e.,  we  test 

Hq  :  Xj  ~  N(0,1) 

versus 

Hj  :  Xj - N(l,l)  ,i=  1,2,3. 
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Assume  that  tcq  =  0.5,  =  Cqq  =  0  and  Cjq  =  Cqi  =  1.  Let  us  consider  the  case  where  the 

local  detectors  are  designed  so  as  to  minimize  their  own  probability  of  error.  It  can  be  eas¬ 
ily  seen  that  the  probability  of  detection  for  each  local  detector  is  P &  =  0.6915  (or  the 
probability  of  miss  P m  -  0.3085)  and  the  probability  of  false  alarm  Pp  =  0.3085.  The  local 
decisions  are  sent  to  the  fusion  center  for  decision  combining  and  for  yielding  a  final  deci¬ 
sion.  The  possible  fusion  rules  for  the  system  are  the  AND  fusion  rule,  the  OR  fusion  rule, 
and  the  majority  logic  rule.  The  probability  of  error  achieved  by  each  one  of  these  fusion 
rules  is  given  by 

P(E)and  =  ^qPf3  +  *iOPm-1Pm2+Pm3)  =  0.41335 


P(E)MAJ  =  KoOPf2-2Pf2)  +7t1(3PA/2-2/V3)  =  0.22679 


P(E)0R  =  nQOPr3PF2+PF2)  +  Tt\PM3  =  0.28534 

As  will  be  seen  later,  applying  equation  (4.4),  the  minimum  probability  of  error  can  be 
computed  to  be 

/>(£)  =  0.5-0.5£  m|n0P*(l-/>f)5-‘-^Po(l-Po)3-*| 
k=  0 


which  results  in 

P(E)  =  0.22679  ■ 

As  can  be  seen  from  Example  4.1,  equation  (4.4)  resulted  in  the  probability  of  error 
(POE)  corresponding  to  the  best  fusion  rule,  namely,  the  majority  logic.  As  illustrated  in 
Section  4.4,  this  fact  can  be  used  to  drastically  reduce  the  complexity  of  the  design  of  op¬ 
timum  decentralized  detection  systems. 

4.3  Performance  of  Decentralized  MAC  Receivers 

In  this  section,  we  investigate  the  performance  of  decentralized  detection  systems  as 
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a  function  of  number  of  detectors  with  minimum  average  cost  as  the  performance  mea¬ 
sure.  Initially,  we  consider  the  case  of  hard  decisions  where  the  local  observations  are 
quantized  into  two  levels  and  the  quantized  value  is  transmitted  to  the  fusion  center.  First, 
results  are  derived  for  a  two-sensor  system  and  then  they  are  extended  to  three  and  n  sen¬ 
sor  systems  respectively.  The  treatment  is  valid  for  any  operating  point  (Pq,  Pp)  on  the  re¬ 
ceiver  operating  characteristic  of  the  local  detectors,  i.e.,  system  is  not  necessarily  the 
optimum  system.  The  results  are  further  generalized  to  the  soft  decision  case,  i.e.,  to  the 
case  with  n  sensors  and  M  quantization  levels  at  the  local  detectors.  Once  again,  the  re¬ 
sults  are  valid  for  any  set  of  local  thresholds  Tq  <  Tj  <  ...  <  T^j  which  are  used  to  perform 
quantization. 

4.3.1  Performance  of  a  Two-sensor  System 

Let  us  consider  a  two-sensor  system.  The  fusion  rules  for  this  structure  are  limited  to 
two,  the  AND  fusion  rule  and  the  OR  fusion  rule  (the  trivial  cases  of  always  deciding  in 
favor  of  Ho  or  always  deciding  in  favor  of  Hj  are  not  considered).  Using  (2.1)  we  compute 
the  system  average  cost  for  each  fusion  rule  as 

rand  =  Coo*0(l-^)  +Cw*<>r2F  +  C01Kl(l-P2D)  +  C117t1/>2 

(4.9) 

ror~  coo*o  ( ^  —  R f)  2  +  c\<Fo  (2Pp  ~  P  p)  +  Coi*!  ( 1  ~  Pq)  2  +  C uTtj  (2 Pp  —  Pq) 

(4.10) 


If  we  denote  by  R2  the  MAC  of  the  two  sensor  system,  then 


R2  -  min  (P/Mp,  Ror) 


(4.11) 


Substituting  (4.9)  and  (4.10)  in  (4.11)  and  using  (2.16)  we  get 
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R2  -  /?!  | ^  1  ^*Af  ( 1  ^Af)  (Coi  ^ll)  n0^F^  Pp)  (ClO 


where 


(4.12) 


^1  “  Coo7l0(l  Pp)  +  CiqKqPf  +  Cq^K^Pm  +  CjjTtj  ( 1  PM) 


The  first  term  on  the  right  hand  side  of  (4.12),  Rj,  represents  the  average  cost  for  a  single 
sensor  system.  The  second  term,  which  is  a  positive  quantity,  represents  the  improvement 
in  performance  when  two  sensors  are  used  with  the  same  cost  assignment  as  before  at  the 
fusion  center.  An  interesting  situation  arises  when 

O  ~Pm)  (Coi”Cn)  =  ~pf)  (Cio~Qx))  (4.13) 


In  thir;  case,  the  improvement  term  in  (4.12)  vanishes  and,  consequently,  the  performance 
of  th  two  sensor  system  reduces  to  that  of  a  single  sensor. 

4.3.2  Performance  of  a  Three-sensor  System 

When  the  number  of  local  detectors  is  three,  the  possible  fusion  rules  for  the  system 
are  the  AND  fusion  rule,  the  MAJORITY  logic  fusion  rule,  and  the  OR  fusion  rule.  These 
fusion  rules  yield  the  following  average  costs 


^  \ND  —  Pp)  +  C^qKqPf+  C0j7tj(l  (1  P  M)  3)  +  Cj  ( 1  P  M) 3 

(4.14) 

^ or  ~  coo*o  ( ^  3  +  C]0^o  (1  ~  ^  —  3)  +  C0ln,PM  +  Cn7tj  (1  -  PM) 

(4.15) 
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^maj  “  QoM1  3P^+2Pp)  +C10tc0(3/>^  2Pp)  +C0'lit)  (3 PM  2 PM) 


+  C11*1(l-3P^  +  2/»Jf)  (4.16) 

As  in  the  two  sensor  case,  the  MAC  for  this  system  is 

/?3  =  miniminiR^Q,  ror)'Rmaj}  (4.17) 

Substituting  (4.14),  (4.15),  and  (4.16)  in  (4.17)  we  get  (see  Appendix  B  for  details) 

r3  =  Rx  -  A  [B  +  3|A|  +  3|B  -  |A||]  =  /?!  -  A13  (4.18) 

where  B  and  IAI  are  given  by 

B  =  1L0PF(l-3PF+2Pj)  (C^-Co,,)  +n]Pu(\-3Pu  +  2Pll)  (C01-C„) 

Ml  =  \*,pu(l-pu)  (Cm-cn)  -n0PF(l-PF)  (Ct0-CK)\ 

The  term  between  brackets  on  the  right  hand  side  of  (4.18)  represents  the  improvement 
that  the  three-sensor  system  has  over  the  single  -sensor  system.  Now  we  show  that  this 
term  is  always  positive  indicating  that  an  improvement  in  system  performance  is  guaran¬ 
teed  as  we  go  from  a  single-sensor  system  to  a  three-sensor  system. 

We  start  first  with  the  interesting  case  where  0  <  Pp  <  0.5  and  0  <  Pj^  <  0.5.  Now 
consider  the  function 
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(4.19) 


u(  x)  =  l-3.r  +  2.r2 

This  function  is  plotted  in  Fig.  4. 1 .  As  is  clear  from  this  figure,  the  function  u(x)  is  positive 
for  values  of  x  in  the  range  0  <  x  <  0.5.  By  examining  the  expression  for  B  in  equation 
(4.18),  we  see  that  both  the  first  and  the  second  term  contain  functions  of  Pp  and  Pm  of  the 
form  (4.19).  Therefore,  for  values  of  Pp  and  PM  in  the  ranges  specified  above,  the  value  of 
B  is  positive.  When  B  is  positive,  all  three  terms  between  brackets  on  the  right  hand  side 
of  (4.18)  are  positive,  indicating  that  the  improvement  term  Aj3  is  positive  and,  therefore, 
it  is  obvious  that  R3  <  Rp  For  other  values  of  Pp  and  PM  that  make  B  negative,  we  make 
the  change  of  variables  B  =  -  D  in  which  D  is  positive.  Now  consider  the  quantity  A13  giv¬ 
en  in  (4.18)  that  represents  the  improvement  in  performance  of  the  three-sensor  system 
over  that  of  the  single-sensor  system.  In  terms  of  IAI  and  D,  the  improvement  A13  can  be 
written  as 


A13  =  ^  [310  +  \A\\  -D  +  3|/t|  | 


(4.20) 


But 


IO  +  IAII  =  D  +  IAI 


(4.21) 


Making  use  of  (4.21), we  sec  that 


AI3  =  ^  [D  +  3|A|] 


(4.22) 


which  clearly  shows  that  A j 3  >  0.  To  examine  whether  a  condition  similar  to  (4  13)  exists 
in  which  the  performance  of  the  three-sensor  system  is  the  same  as  the  performance  of  a 
single-sensor  system,  we  need  the  improvement  term  A j 3  in  (4.22)  to  be  equal  to  O.This 
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happens  when  IAI  =  0  and  D  =  -  B  =  0.  The  condition  IAI  =  0  leads  to  the  requirement  that 

—  ^*Af)  (^oi  —  )  =  ko^f  —  (^io  ~  Q)o)  (4.23) 

while  the  condition  B  =  0  requires  that 

Vf  <1  -  3pF  +  2/>|>  (C,o  -  Coo)  =-*/«(!-  +  2P l)  (C01  -  C„ ) 

(4.24) 

Equation  (4.24)  can  also  be  rewritten  as 

W  (1-^)0-  2 Pp)  (C10  -  Coo)  =  -*1  PM  ( 1  -  ■ PM)  0 1  -  2 PM)  (C01  -  Cn ) 

(4.25) 

Substituting  (4.23)  into  (4.25)  we  get 

*o PfO-Pp)  (1-2 Pf)  (C.o-Coo)  =  -WO-Pf)  (l-2/’„)  (Cjo-Coo) 

(4.26) 

By  cancelling  common  terms  on  both  sides  of  (4.26)  we  obtain 

d-2  PF)  =  —  ( 1  -  2P  M)  (4.27) 

Simplifying  (4.27)  we  get  a  relationship  between  Pp  and  as 

PF  ~  1  ~  pm  =  pd  (4.28) 

Therefore,  a  simultaneous  solution  to  (4.23)  and  (4.24)  exists  only  when  Pp  =  PD  which  by 
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the  virtue  of  (4.23)  requires  that 


*1  (Coi  ^ll)  ~  n0  ^10  ^'00^ 


(4.29) 


As  is  well  known  from  detection  theory  [1],  the  requirement  Pp  =  Pq  in  (4.28)  contradicts 
the  concavity  property  of  receiver  operating  characteristics  of  optimum  receivers.  There¬ 
fore,  a  simultaneous  solution  to  (4.23)  and  (4.24)  does  not  exist  and,  consequently,  we 
conclude  that  the  improvement  term  in  (4. 18)  and  (4.20)  cannot  vanish  for  all  values  of  Pp 
in  the  range  0  <  Pp>  <  1  and  Pp  in  the  range  0  <  Pp  <  1  and  a  three-sensor  system  is  always 
superior  to  a  single  sensor  system. 

It  is  also  of  interest  to  compare  the  performance  of  the  three-sensor  system  to  the 
performance  of  the  two-sensor  system.  In  terms  of  the  quantities  IAI  and  B  introduced  in 
(4.18),  we  can  express  (4.12)  as 


R2  = 


(4.30) 


Denote  by  A23  the  improvement  in  performance  of  the  three-sensor  system  over  the  two¬ 
sensor  system.  Subtracting  (4.18)  from  (4.30)  we  get 

A23  =  R2-R3  =  ^  [3||/4|  -B\  -  (|A|  -5)1  (4.31) 

It  is  evident  from  (4.31)  that  A23  >  0.  The  improvement  term  A23  vanishes  when  IAI  =  B. 
This  happens  in  two  cases 

Case  1.  A  =  B 

Using  the  values  of  IAI  and  B  defined  in  (4. 1 8)  we  get  the  condition 
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(4.32) 


MQi-Cn)  _  pfv-pf)2 
Mcio-coo)  ~  PD(l~PD)2 

Case  2.  -A  =  B 

This  condition  leads  to  the  requirement 

^1  (Cqi  -  Cji)  p'p  ( 1  —  P p) 

^o(Cio-Coo)  “  P2D(l-PD) 


We  conclude  that  if  either  condition  (4.32)  or  (4.33)  is  satisfied,  then,  the  perfor¬ 
mance  of  the  three-sensor  system  is  identical  to  the  performance  of  the  two-sensor  system. 
Otherwise,  the  three-sensor  system  outperforms  the  two-sensor  system. 


4.3.3  Performance  of  an  n-sensor  System 

For  n  >  3  listing  all  possible  fusion  rules  and  repeated  application  of  (2. 16)  to  find 
the  minimum  average  cost  of  the  system  using  the  best  fusion  rule  becomes  tedious.  How¬ 
ever,  we  are  able  to  find  the  minimum  average  cost  of  the  system  with  the  best  fusion  rule 
in  an  alternative  way.  Recall  that  zj,  the  decision  of  local  detector  LDj  has  a  Bernoulli  dis¬ 
tribution  with  parameter  Pp  when  Hq  is  true  and  parameter  Pj>  when  Hj  is  true.  Under  our 
assumption  of  independence  and  identical  sensors,  the  vector  U  is  a  sequence  of  Bernoulli 
trials.  If  K  represents  the  number  of  sensors  that  decide  in  favor  of  Hj,  i.e.,  the  number  of 
l’s  in  the  vector  U,  then  K  has  the  following  conditional  distributions 


P  oW 


Pp  ( 1  “  PF) 


n  —  k 


(4.34) 


Pi  (*) 


(1  -pd)h 


k 


(4.35) 


The  system  minimum  average  cost  based  on  the  vector  U  is  found  with  the  aid  of  (2.22)  as 
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«n  =  *0-0-5  2  m|c,p‘  (l-l«D)"-‘-C^ (1  -PF)"-*|  (4.36) 

k  =  0 

Equation  (4.36)  is  an  explicit  relationship  between  the  system  MAC  and  the  number  of 
sensors.  It  also  represents  the  minimum  system  average  cost  corresponding  to  the  best  fu¬ 
sion  rule  implemented  by  the  fusion  center.  Let  m  be  an  integer  larger  than  n,  then  the 
quantity  Rn  -  Rm  represents  the  improvement  in  system  perfofmance  when  (m-n)  new  sen¬ 
sors  are  added  to  a  system  originally  having  n  sensors. 


4.3.4  Performance  of  an  M-level  Quantized  System 

In  the  preceding  subsection,  the  performance  evaluation  was  carried  out  for  the  case 
when  LD;  transmits  a  single  hard  decision  to  the  fusion  center.  In  that  case  the  probabili¬ 
ties  of  detection  and  false  alarm  can  be  appropriately  defined.  In  this  section  we  will  gen¬ 
eralize  the  results  to  the  case  where  local  detector  LDj  transmits  one  of  the  M  possible  soft 
decisions  to  the  fusion  center. 

Recall  from  Section  4.1  that  for  the  case  M  >  2,  the  vector  U  is  a  sequence  of  n  i.i.d 
generalized  Bernoulli  trials.  Let  Xk,  1  <kSM,  represent  the  number  of  sensors  that  have 
decided  in  favor  of  symbol  ak.  The  number  of  such  symbols  in  U  follows  the  following 
conditional  densities 


f 


P  (X:-  xv  H0)  - 


n 


\XlX2..-XMjTl.P 

k  =  1 


M 


k 

kO 


(4.37) 


P  (X]-  xv  xM  Hx)  - 


M 


[xlx2...xM 


JIP« 

k=  1 


(4.38) 


where 
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n 


xtx?...x. 


Vll-2 


MJ 


n! 


xllx2\...xM\ 


M 

X  i*  = n 

*  -  1 

and  Pj-j  ;  j=0, 1  in  the  case  of  identical  sensors  are  given  by  (4. 1  ).The  MAC  for  this  system 
is  found  using  (2.21)  as 


/?a/('0  /?„  X  (a1.t2....xAf 


M  x  -W  x 

c.n^-con^! 

k  =  1  k  =  1 


(4.39) 


The  summation  on  the  right  hand  side  of  (4.39)  is  taker.  ■>  r  all  possible  values  of  .\j,  \2  , 
x^  such  that  Xj+X2+...+xM  =  n.  Analogous  to  the  case  M  =  2,  the  quantity  Rvl(n)  - 
Ryi(m)  represents  the  improvement  in  the  system  performance  when  (tn-n)  new  local  de¬ 
tectors  are  added  to  a  system  originally  composed  of  n  sensors. 

4.4  Design  of  the  Optimum  Decentralized  MAC  Detection  System 

The  two  major  problems  encountered  in  the  design  of  a  decentralized  detection  sys¬ 
tem  with  data  fusion  for  a  given  number  of  ’  ^al  detectors  are  the  determination  of  the  op¬ 
timum  fusion  rule  and  the  optimum  local  decision  rules.  For  tlie  first  part  of  the  design 
problem,  we  assume  the  fusion  center  to  be  a  minimum  average  cost  receiver  with  (4.2)  as 
the  decision  rule.  This  decision  rule  when  implemented  at  the  fusion  center  optimally  par¬ 
titions  the  observation  space  of  the  decision  vector  U  into  two  mutually  exclusive  decision 
regions  corresponding  to  hypotheses  Hq  and  Hj.  This  partitioning  yields  the  smallest 
achievable  average  cost  for  any  given  arbitrary  local  decision  rules.  The  partitionning  cor¬ 
responds  to  a  specific  fusion  rule  (  see  Appendix  A  for  the  algorithm  used  by  the  fusion 
center  to  combine  the  local  decisions  and  to  make  the  global  decision)  In  the  previous 
section  wc  derived  the  expression  for  the  minimum  average  cost  for  an  n  sensor  system 


with  two  quantization  levels  as  well  as  M  quantization  levels.  For  the  second  part  of  the 
design  problem,  our  objective  is  to  select  the  local  decision  rules  such  that  the  average  cost 
is  globally  minimized  resulting  in  the  optimum  decentralized  MAC  system. 

Let  T  =  be  the  vector  consisting  of  local  thresholds  at  any  of  the 

identical  detectors.  The  probabilities  Pjy,  j  =  0,  1,  given  by  (4.1)  are  functions  of  the 
thresholds.  We  stress  this  by  expressing  Pkg  and  P^j  as 

?  kj  =  ^kj  (Tfr  Tk_l),j  =  0,  1  (4.40) 


for  1  <  k  <  M.  For  a  given  threshold  vector  X  the  global  decision  rule  (4.2)  expressed  as  an 
explicit  function  of  T  becomes 


uoCD= 


{ 


1  if  A(T)  >  0 
0  otherwise 


(4.41) 


where 


A  (I)  =  ^ - 

rK«.<r>|«o> 


For  a  given  value  of  Xthe  decision  rule  (4.41)  yields  the  smallest  average  cost  among  all 
other  decision  rules  (fusion  rules).  The  resulting  system  MAC  as  a  function  of  T  can  be 
expressed  as 

m  x  m  x 

C,  n  pk\  (Tt.  Tk_ ,)  -  C0  n  P*  (Tt,  Tk_ ,) 

k=  1  k=  1 

It  should  be  reemphasized  that,  for  a  fixed  X>  if  one  evaluates  the  system  average  cost  for 
all  possible  fusion  rules,  then  (4.42)  gives  the  minimum  among  all  of  the  above  system  av- 
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crage  costs  and  the  fusion  rule  which  yields  the  minimum  is  specified  by  (4.41).  For  a  dif¬ 
ferent  value  of  X  say  T",  the  entire  process  needs  to  be  repeated,  i.e.,  equations  (4.42) 
and  (4.41)  provide  the  minimum  system  average  cost  for  the  local  threshold  vector  T",  and 
the  corresponding  fusion  rule  respectively.  Therefore,  to  design  the  overall  system, 
R-MnCD  should  be  minimized  with  respect  to  T  and  the  corresponding  fusion  rule  can  be 
determined  from  (4.41  ).  The  resulting  system,  i.e.,  the  resulting  value  of  T  and  the  corre¬ 
sponding  fusion  rule,  yields  the  minimum  achievable  cost  for  the  system. 

Note  that  the  above  design  procedure  requires  the  minimization  of  a  single  function 
of  (M-l)  variables.  Also,  even  when  the  assumption  of  identical  thresholds  at  the  local  de¬ 
tectors  is  relaxed,  we  need  to  minimize  a  single  function  of  n(M-l)  variables.  This  means 
that  for  the  binary  case  (M=2),  we  need  to  minimize  a  function  of  n  variables  only.  This 
case  of  non  identical  thresholds  will  be  considered  in  the  next  chapter.  Our  procedure  is 
computationally  simpler  than  the  design  procedures  available  in  the  literature  where  opti¬ 
mization  needs  to  be  carried  out  for  all  possible  fusion  rules.  Two  methods  that  deal  with 
the  design  of  binary  decentralized  Bayesian  detection  systems  are  reported  in  the  litera¬ 
ture.  In  the  first  method  [12],  the  fusion  center  is  fixed  and  a  set  of  n  coupled  nonlinear 
equations  are  solved  to  determine  the  n  local  thresholds.  This  has  to  be  repeated  for  all  the 
permissible  fusion  rules.  The  solution  with  the  smallest  overall  cost  is  finally  selected  as 
the  optimum  system.  The  exponential  growth  of  the  number  of  fusion  rules  to  be  searched 
makes  the  use  of  this  method  impractical.  The  other  method  is  the  person-by-person  opti¬ 
mization  procedure  [10]  where  conditions  are  determined  for  each  local  detector  and  the 
fusion  center  so  as  to  minimize  the  system  cost  when  the  other  system  components  are  as¬ 
sumed  to  be  fixed.  The  resulting  equations  are  solved  simultaneously  to  yield  the  person- 
by-person  optimal  solution.  This  solution  is  not  necessarily  the  optimum  solution.  This 
procedure  requires  a  simultaneous  solution  of  (2n+n)  coupled  nonlinear  equations  for  the 
binary  Hypothesis  testing  problem.  Thus,  our  design  procedure  is  computationally  efficient 
and  yields  the  optimum  solution.  It  can  be  employed  for  the  design  and  performance  eval¬ 
uation  of  relatively  large  detection  networks  as  shown  in  the  next  chapter. 

In  our  design  procedure  when  M  =  2,  the  n-dimensional  vector  T  reduces  to  a  scalar. 
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In  this  case  the  local  detectors  perform  their  likelihood  ratio  tests  with  respect  to  a  com¬ 
mon  threshold  x  (see  equation  4.5).  For  a  given  value  of  x,  the  local  detection  probability 
Pj5  and  the  local  false  alarm  probability  Pp  are  functions  of  x.  We  emphasize  this  by  ex¬ 
pressing  Pjj  as  PD(x)  and  Pp  as  Pp(x).  Therefore,  we  can  express  the  conditional  distribu¬ 
tions  given  by  (4.6)  and  (4.7)  of  the  local  decision  Zj  in  terms  of  x  as 

/0(Zi(t))  =  [Pri-DYi  [l-Pp(x)]1-2/  (4.43) 

/l(Zj(x))  =  [PD(.z)]zi  [l-PD(x)]l'zi  (4.44) 

The  conditional  densities  of  the  decision  vector  U  correspondingly  can  be  expressed  as 

n 

fj(Uix))  =  m  (z,(x)),;  =  0,  1  (4.45) 

i  =  I 

The  likelihood  function  in  (4.41)  expressed  as  a  function  of  X  is 

fx{U  00) 

*(ffW)=5M  <446> 

As  pointed  out  in  remark  2  in  Section  4.2,  the  decision  rule  (4.8)  with  A(U(x))  as  given  in 
(4.46)  specifies  the  fusion  rule  with  the  smallest  MAC  for  a  given  value  of  x.  The  MAC 
corresponding  to  this  fusion  rule  with  n  incoming  decisions  expressed  in  terms  of  x  is 

=  *0-5  S  (tj|C1/>D(t)<l-/’D(T))"-*-C0^(t)  (l-/>f(T))"'‘| 
k  =  0 

(4.47) 

To  obtain  the  system  with  global  optimum  cost  (least  achievable  MAC),  Rn(x)  should  be 
minimized  with  respect  to  the  threshold  x.  The  resulting  local  threshold  alongwith  the  cor¬ 
responding  fusion  rule  specified  by  (4.46)  and  (4.41)  will  yield  the  optimum  system. 
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4.5  Minimum  Probability  of  Error  Systems 


In  this  section  we  consider  the  design  of  the  global  minimum  probability  of  error 
(MPOE)  systems  which  is  a  special  case  of  the  results  obtained  in  the  previous  section.  In 
addition,  we  consider  the  design  of  several  suboptimum  decentralized  detection  systems 
and  compare  their  performances.  The  first  suboptimum  system  is  based  on  local  optimiza¬ 
tion  in  that  each  system  component  is  an  MAP  receiver.  The  next  two  suboptimum  sys¬ 
tems  are  based  on  the  discrimination  and  the  Bhattacharyya  distance  measures. While 
these  two  measures  have  been  utilized  before  in  the  design  of  decentralized  detection  sys¬ 
tems  [35,38],  here  we  employ  them  in  a  different  manner.  Finally,  we  present  a  subopti¬ 
mum  design  procedure  based  on  our  new  tight  upper  bound  on  the  probability  of  error 
developed  in  Chapter  3.  For  simplicity,  we  restrict  our  attention  to  the  case  M  =  2  in  this 
section.  We  conclude  this  section  with  two  numerical  examples  that  compare  the  perfor¬ 
mance  of  the  optimum  system  to  the  performances  of  the  various  suboptimum  systems. 


4.5.1  Global  MPOE  System  Design 

t 

As  an  important  application  of  the  general  Bayesian  formulation  developed  in  the 
previous  section,  we  consider  the  cost  assignment  C()()=  Cn  =  0  and  C01  =  Cl0  =  1.  This 
corresponds  to  the  minimum  probability  of  error  criterion  which  is  widely  used.  When  the 
decentralized  system  is  to  be  designed  using  the  probability  of  error  as  the  system  perfor¬ 
mance  measure,  the  expression  for  the  minimum  probability  of  error  as  a  function  of  the 
local  threshold  x  is  given  by 


pc(x)  =  0.5- ^  £ 


. 

n 

2  ^  \k) 

k  =  0 


7t,^(X)  (l-Po(l))n~k-K0Pk  (X)  (\-PF{X))n- 


D 


(4.48) 


To  design  the  global  MPOE  system,  we  need  to  determine  the  value  of  x  that  minimizes 
Pe(t)  given  by  (4.48)  and  then  determine  the  fusion  rule  which  corresponds  to  that  value 
of  the  probability  of  error  using  (4.41)  (see  also  Appendix  A). 


7<; 


4.5.2  System  Design  Based  on  Local  Optimization 


In  this  section  we  assume  that  the  strategy  of  the  local  detectors  is  to  select  the 
threshold  that  minimizes  the  POE  at.  the  local  level.Let  po(xj)  and  Pi(xj)  be  the  conditional 
densities  of  the  observation  Xj  received  by  local  detector  LDj.  The  decision  rule  (4.5)  that 
determines  the  decisions  zj  becomes 


Zi 


■{ 


1  if  PiCxjVpoCxj) 


(4.49) 


0  otherwise 

It  is  clear  from  (4.49)  that  the  threshold  Tl=  and  t^ie  system  POE  is  Pe(Tl).  When 
decision  rule  (4.49)  is  implemented  at  the  local  level,  all  the  system  components  in  Fig.  1. 1 
become  MAP  receivers.  Even  though  each  component  is  individually  optimized,  the  sys¬ 
tem  as  a  whole  can  be  far  from  optimum,  since  the  sensor  decisions  are  made  independent¬ 
ly  of  the  global  decision. 


4.5 3  System  Design  Based  on  Discrimination 

Here,  we  describe  the  design  procedure  based  on  the  discrimination  measure.  The 
discrimination  between  two  conditional  probability  density  functions  f0(x)  and  fj(x)  of  a 
continuous  random  variable  X  is  defined  as 

r  /o  (■*) 

J  =  J/o  U)  ln]^Jx)dx  (4.50) 


When  the  random  variable  X  is  discrete, the  integration  in  (4.50)  is  replaced  by  a  summa¬ 
tion  and  the  probability  density  functions  are  replaced  by  probability  mass  functions.  In 
this  case  the  discrimination  J  becomes 

^  P  (X=  *1 H0) 

J.'ZPV.Wb,-—'.  (4.5,) 


Several  important  properties  of  the  discrimination  measure  are  listed  in  [67],  Since  the  fi- 
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nal  decision  in  the  decentralized  detection  system  is  based  on  the  vector  U=[Z],Z2,...,znJ, 
we  will  examine  the  discrimination  between  the  conditional  distributions  of  the  vector  U 
under  the  two  hypotheses.  This  is  given  by 


K  (I’d-  pr)  =  I1P  (U=  *1 
u 


~zn\  H0)  In 


P(U=  zv..2n\H  o) 
P(f/=  zv-zn\Hx) 


(4.52) 


The  local  decisions  z,  are  Bernoulli  random  variables  with  conditional  density  functions 
given  by  (4.43)  and  (4.44).  Recognizing  that  U  is  a  sequence  of  n  independent  and  identi¬ 
cal  observations,  then  using  the  additivity  property  of  the  discrimination  measure  we  get 

Jn(PDW’PFW)  =  nJ^X)  (4.53) 


Here,  Jj(x)  is  the  discrimination  between  the  conditional  distributions  of  a  single  local  de¬ 
cision  zj  under  the  two  hypotheses.  Using  (4.43)  and  (4.44)  we  get 


P  r(T)  (1  -PF(X)) 

Jx  (X)  -  PF  (x)  lnpD(x)  +  ( 1  -  Pp  (X) )  In  (1  -pD(x)) 


(4.54) 


As  an  explicit  function  of  the  local  threshold  x,  the  discrimination  between  the  conditional 
densities  of  U  can  be  expressed  as 


n 


PF  (x)  In 


PF(X) 

PDW 


+ 


(1  -PF{x))ln 


(1  -PF(x))l 
U-PD(x))_ 


(4.55) 


By  making  use  of  the  Blackwell  theorem  we  observe  that  an  indirect  way  of  minimizing 
the  POE  is  to  maximize  the  discrimination.Denote  by  x^  the  specific  value  of  the  local 
threshold  x  that  maximizes  Jn(x).  The  system  POE  in  this  case  is  Pe(*d)  where  Pg(x)  is 
given  by  (4.48).Two  factors  make  this  system  suboptimum.  First,  The  discrimination  as 
defined  in  (4.50)  is  independent  of  the  prior  probabilities.  This  fixes  the  local  thresholds 
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and  consequently  the  values  of  Pp  and  Pq  to  be  used  during  the  system  design.  Designing 
the  system  based  on  this  measure  when  the  prior  probabilities  are  known,  in  fact,  means 
that  part  of  the  information  available  about  the  system  is  not  exploited.  Second,  the  dis¬ 
crimination  measure  is  independent  of  the  fusion  rule  and,  therefore,  it  is  not  a  pan  of  the 
overall  optimization  process.  This  leads  to  the  suboptimum  system. 

4.5.4  System  Design  Based  on  Bhattacharyya  Distance 

Now,  we  consider  the  design  procedure  based  on  the  Bhattacharyya  distance.  Let  Xj 
be  the  random  observation  in  a  hypothesis  testing  problem  with  corresponding  conditional 
densities  fo(xi)  and  fiCx^.  The  Bhattacharyya  coefficient  is  given  by 


Pi  =  (x\)dxx  (4.56) 

*1 

When  the  random  variable  Xt  is  discrete,  the  Bhattacharyya  coefficient  is  defined  as 

P,  =  £.//>(*,=  (4.57) 

*1 


The  Bhattacharyya  distance  (see  Chapter  1)  is  defined  as 

Dj  =  -In  pj=  ( xx)dxx  (4.58) 

If  X  is  a  sequence  of  n  independent  and  identically  distributed  observations,  then  the  Bhat¬ 
tacharyya  coefficient  and  the  Bhattacharyya  distance  corresponding  to  the  conditional 
densities  of  X  become 


(4.59) 
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Pn  =  -nln  pj 


(4.60) 


The  Bhattacharyya  coefficient  for  the  conditional  densities  of  U  is 


p n(PPrD)  =  IJWm  TpW  z,...2„| w,))  (4.6D 

U 


The  Bhattacharyya  distance  between  the  conditional  densities  of  the  local  decision  Zj  is 


D^Pp.Po)  =  -ln[JP^,  +  J(\-PF)  (1  ~PD)]  (4.62) 


Expressing  PD  as  PD(x)  and  Pp  as  Pp(x)  and  making  use  of  (4.60)  we  can  express  the  Bhat¬ 
tacharyya  distance  between  the  conditional  densities  of  U  in  terms  of  x  as 

D„(t)  =  -nln[JPF(z)PD(x)+JO-PFW)  (l-M*))]  (4.63) 


Again,  making  use  of  the  Blackwell  theorem  we  see  that  an  indirect  way  of  minimizing 
the  POE  is  to  maximize  the  Bhattacharyya  distance.  Denote  by  xB  the  specific  value  of  x 
that  maximizes  Dn(x).  The  system  POE  in  this  case  is  Pp(xB).  Again,  the  resulting  system 
is  suboptimum  and  the  reasons  for  this  are  the  same  ones  as  discussed  in  the  previous  sub¬ 
section. 

4.5.5  System  Design  Based  on  the  New  Upper  Bound  on  the  MPOE 

In  this  section  we  apply  the  new  upper  bound  derived  in  Chapter  3  to  the  design  of 
decentralized  detection  systems.  Let  K  be  the  random  variable  representing  the  number  of 
local  detectors  that  decide  in  favor  of  hypothesis  Hj.  Then  K  has  the  conditional  density 
functions  given  by  (4.34)  and  (4.35)  under  hypotheses  Hq  and  Hj  respectively.  Since  K  is 
a  discrete  random  variable  then,  in  order  to  find  an  upper  bound  on  the  probability  that  the 
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fusion  center  makes  an  error,  we  need  to  express  the  new  upper  bound  given  by  (3.7 1 )  in  a 
discrete  form.  This  is  given  as 


P(E)<  £  (0.5  (sinnpk)exp  [-1.8063  (p*-  0.5 )' 2] )  (n0P0  (k)  +  nlPl  (k)) 

(4.64) 


k  =  0 


where 


Pk  = 


Vo 


+  V*i  (*) 


,  £  =  0,  1, n 


is  the  a  posteriori  probability  of  hypothesis  Hq  given  that  K  =  k.  Substituting  (4.34)  and 
(4.35)  into  the  above  expression  for  p^,  we  get 

iz0PkF(l-PF)n~k 

Pk=  - j - — r - t - r,*  =  0,  1,  ...,n  (4.65) 

n0PkF(l-PF)n  k  +  nlPkD(l-PD)n 

The  upper  bound  in  (4.64)  applies  for  any  arbitrary  point  (Pp,  PD)  on  the  receiver  operat¬ 
ing  characteristic  of  the  local  detectors,  i.e.,  for  any  value  of  the  local  threshold.  Let  x  be 
the  common  threshold  of  the  local  detectors  (see  equation  (4.5))  Expressing  Pp  as  Pp(x) 
and  PD  as  PD(x),  we  can  write  the  a  posteriori  probabilities  p^  in  terms  of  x  as 


PkW  = 


n0(PF(x))k(\-PF(x))n~k 


it0(PF(x))k{l-PF(x))n~k  +  Kx  ( PD(x))k(l-PD(x))n~k 


(4.66) 


The  upper  bound  on  the  MPOE  in  terms  of  x  becomes 

P(E)  <,  £  0.5  (j*)sinjv*(x)  (exp  [-1.8063  (p*(x)  -0.5) 2] ) 


*  =  o 


X  [K0(PF(X))k(\-PF(X))n-k+K](PD(x))k(\-PD(x))n-k} 


(4.67) 
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The  design  of  the  system  using  the  new  upper  bound  calls  for  determining  the  local  thresh¬ 
old  x*  which  minimizes  the  right  hand  side  of  (4.67).  Therefore,  a  suboptimum  system 
that  is  designed  to  minimize  the  upper  bound  on  the  probability  of  error  is  one  in  which 
the  local  detectors  use  the  threshold  t*.  The  resulting  POE  is  given  by  PgCc*).  Because  of 
the  tightness  of  the  new  upper  bound,  systems  using  this  bound  as  the  design  criterion  are 
nearly  optimum.  Example  4.3  demonstrates  the  utility  of  the  new  upper  bound  in  the  de¬ 
sign  of  decentralized  detection  systems.  Note  that  the  knowledge  of  the  prior  probabilities 
is  used  in  this  approach  but  it  is  still  independent  of  the  fusion  rule. 

4.5.6  Examples 

In  this  subsection,  we  present  two  numerical  examples  where  the  system  perfor¬ 
mances  achieved  by  using  the  different  design  approaches  is  compared. 

Example  4.2 

Consider  a  two  sensor  decentralized  detection  system  used  for  the  detection  of  a  con¬ 
stant  signal  of  level  m  embedded  in  a  zero-mean  Gaussian  noise  with  variance  a2.  The 
null  hypothesis  corresponding  to  noise  alone  and  the  alternative  hypothesis  corresponding 
to  signal  plus  noise  are  expressed  as 

Hq  :  X,  ~  N(0,  ct2) 

H!  :  Xj  ~  N(m,  d2) 

For  this  binary  hypothesis  testing  problem  we  will  compare  the  performance  of  the 
various  suboptimum  systems  considered  previously  to  the  performance  of  the  optimum 
decentralized  system  as  well  as  the  optimum  centralized  system.  Also,  we  will  see  how  the 
performance  of  the  optimum  decentralized  system  improves  as  the  number  of  quantization 
levels  increases.  For  the  one  dimensional  Gaussian  problem  the  likelihood  ratio  test  (4.5) 
at  the  local  level  becomes 
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where 


Hi 

Pl(*i)  > 


Po(*i)  < 
Ho 


Pi  (*,•) 


&exp 


(Xj-m)2 

2c2 


(4.68) 


(4.69) 


Po  U.) 


(470) 


Substituting  (4.69)  and  (4.70)  into  (4.68)  and  simplifying  we  get  the  equivalent  test 

Hi 

> 

Xj  t^/mlog  z+m/2  =T  (4.71) 

< 

Ho 


In  this  and  future  examples  we  find  it  more  useful  to  evaluate  the  equivalent  threshold  T 
defined  in  (4.71)  than  to  evaluate  the  threshold  x.  This  is  because  (4.71)  gives  us  the  deci¬ 
sion  regions  corresponding  to  Hq  and  Hj  in  terms  of  the  observation  x,  while  (4.68)  gives 
the  decision  regions  in  terms  of  the  likelihood  function.The  probabilities  of  false  alarm 
and  detection  expressed  in  terms  of  x  are  given  by 

PF  (?)  =  erfc  (oi~  +  ^)  (4.72) 


PD(z) 


(4.73) 
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where  erfc  (u)  is  the  complementary  error  function  defined  as 


erfc(u)  =  j-jLexp{~)du  (4.74) 

For  parameter  values  m  =  1  and  =  1,  the  design  is  carried  out  for  all  values  of  the  a  pri¬ 
ori  probability  tcq.  The  optimum  decentralized  MPOE  system  with  M  >  2  is  designed  ac¬ 
cording  to  (4.42),  while  for  the  case  M  =  2,  the  system  is  designed  according  to  (4.48). 
The  suboptimum  systems  based  on  the  optimization  of  the  POE  at  the  local  levels,  maxi¬ 
mum  discrimination,  and  maximum  Bhattacharyya  distance  are  designed  according  to 
(4.49),  (4.55), and  (4.63)  respectively.  In  Fig.  4.2,  we  present  the  performance  of  the  vari¬ 
ous  suboptimum  systems  as  compared  to  the  optimum  decentralized  system  and  the  opti¬ 
mum  centralized  system.  Several  observations  can  be  drawn  from  this  graph 

1)  The  graph  shows  clearly  that  the  optimum  decentralized  system  significantly  outper¬ 
forms  all  three  suboptimum  systems  over  a  wide  range  of  ag. 

2)  Of  special  interest  is  the  point  with  7tQ  =  0.5.At  this  point  the  local  decision  rule  (4.49) 
becomes 

{1  if  Pi(xi)/p0(xi)  >1 

(4.75) 

0  otherwise 

Using  (4.75)  we  find  that  the  threshold  Tl=1  and  the  corresponding  local  threshold  in 
terms  of  the  observations  Xj  is  TL  =  0.5  (see  equation  (4.71)).  The  values  of  PD  and  PF  are 
0.6915  and  0.3085  respectively.  The  POE  for  any  one  of  the  local  detectors  is  Pj(E)  = 
0.3085.  Using  (4.48)  we  find  the  POE  of  the  two-sensor  system  to  be  PF(x  =  1)  or  Pe(Tl  = 
0.5)  =  0.3085.  These  results  indicate  that  a  single  sensor  system  ind  a  decentralized  two- 
sensor  system  are  identical  in  terms  of  performance.lt  is  to  be  noted  that  in  this  case  the 
point  (PpPo)  satisfies  condition  (4.13). 

3)  In  Fig.  4.2,  we  notice  that  over  some  range  of  jiq  the  performance  of  the  system  de¬ 
signed  based  on  the  Bhattacharyya  distance  is  better  than  that  designed  based  on  the  dis 
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Fig.  4.2.  Performance  comparison  of  the  optimum  decentralized  system  and  the  subopti¬ 
mum  systems  when  each  local  detector  processes  one  observation. 
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crimination,  and  over  another  range  the  opposite  is  true.  In  fact,  this  is  consistent  with  the 
observations  made  in  [31,32]  regarding  the  selection  of  the  "best"  distance  measure. 

4)  The  performance  curves  for  the  systems  employing  maximum  discrimination  and  max¬ 
imum  Bhattacharyya  distance  have  straight  line  segments  with  different  slopes.  This  can 
best  be  explained  by  substituting  Coo  =  Cli  =  0  and  Ciq  =  Q)i  =  1  into  (4.9)  and  (4. 10)  and 
rewriting  them  as  functions  of  Hq.  The  POE  corresponding  to  the  AND  fusion  rule  and  the 
OR  fusion  rule  are,  therefore,  expressed  as 

P  AND  =  <PF  +  Ph-  2/>M>  *0  +  (2PM  -  fi)  (4-76) 


P  <£)  OR  -  &PF  -  Pl  ~  Ph  *0  +  PM  (4.77) 

As  pointed  out  earlier,  when  we  design  the  system  based  on  members  of  the  class  of  Ali- 
Silvey  distance  measures,  we  find  the  local  thresholds  that  maximize  the  distance  between 
the  conditional  densities  of  the  decision  vector  U.  These  local  thresholds  are  independent 
of  the  prior  probabilities  and,  consequently,  the  values  of  Pp  and  Pj>  of  the  individual  sen¬ 
sors  are  independent  of  these  probabilities.  This  means  that  both  (4.76)  and  (4.77)  reduce 
to  linear  functions  of  Jig.  For  a  given  value  of  Jtg,  the  fusion  center  computes  the  probabil¬ 
ity  of  error  corresponding  to  each  one  of  the  fusion  rules  and  selects  the  fusion  rule  having 
the  smallest  POE.  Since  the  fusion  center  is  a  MPOE  receiver,  we  see  that  over  some  range 
of  tcq,  the  AND  fusion  rule  is  implemented  and  over  another  range,  the  OR  fusion  rule  is 
implemented  depending  on  which  fusion  rule  has  the  smaller  POE. 

Fig.  4.3  shows  the  same  kind  of  comparison  considered  above  but  assuming  that 
each  local  detector  processes  two  observations  to  come  up  with  the  local  decision  Zj.  As 
can  be  seen  from  this  figure,  the  relative  gap  between  the  optimum  system  and  the  subop¬ 
timum  systems  widens  for  0  <  izq  <  1.  For  the  rest  of  the  figures,  single  observations  at  the 
sensors  are  assumed.  In  Fig.  4.4,  we  plot  the  optimum  local  threshold  given  by  (4.71)  ver 
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Fig.  4.3.  Performance  comparison  of  the  optimum  decentralized  system  and  the  subopti¬ 
mum  systems  when  each  local  detector  processes  two  observation. 
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Fig.  4.4.  The  optimum  local  threshold  in  a  two-sensor  decentralized  detection  system. 


sus  %q.  In  Fig.  4.5,  we  plot  the  optimum  fusion  rule  for  the  optimum  decentralized  detec¬ 
tion  system  as  a  function  of  Kq.  As  can  be  observed  from  Fig.  4.5,  for  tiq  <  0.5  the  OR  fu¬ 
sion  rule  performs  better  than  the  AND  fusion  rule  and  for  tcq  >  0.5  the  reverse  is  true.  The 
abrupt  change  in  the  value  of  the  optimum  local  threshold  in  Fig.  4.4  is  a  manifestation  of 
the  fact  that  the  system  always  selects  the  fusion  rule  leading  to  the  smallest  cost.  The 
variation  of  the  optimum  local  threshold  given  by  (4.71)  and  the  optimum  fusion  rule  as  a 
function  of  Jtg  for  the  optimum  three-sensor  decentralized  system  is  shown  in  Figures  4.6 
and  4.7  respectively.  In  this  case  the  majority  logic  has  the  best  performance  over  most  of 
the  region  of  7tg,  i.e.,  for  0.095  <Hq<  0.905.  In  Fig.  4.8,  we  compare  the  performance  of 
the  optimum  decentralized  MFOE  system  with  M  =  2,  3  and  4  to  the  performance  of  the 
optimum  centralized  system.  At  the  point  Ttg  =  0.5  we  observe  that  the  POE  of  the  opti¬ 
mum  decentralized  system  with  M  =  4  is  only  2.36%  higher  than  the  POE  of  the  optimum 
centralized  system.  In  Fig.  4.9,  we  show  the  resulting  MPOE  as  a  function  of  the  signal 
level  m  (small  values  of  m)  for  the  four  systems  when  7t $  =  0.5  is  assumed. 

Example  4.3 

The  objective  of  this  example  is  to  design  the  decentralized  system  based  on  the  new 
upper  bound  for  the  same  problem  as  considered  in  Example  4.2  and  compare  the  perfor¬ 
mance  with  that  of  the  optimum  decentralized  detection  system.  We  further  assume  that  m 
=  1.5.  The  probabilities  of  false  alarm  and  detection  expressed  in  terms  of  x  are  given  by 
(4.72)  and  (4.73).  The  design  equation  for  this  system  will  be  (4.67)  with  n  =  2.  In  Fig. 

4. 10,  we  plot  the  probability  of  error  for  the  system  resulting  from  the  minimization  of  the 
ew  upper  bound  as  a  function  of  (dotted  curve).  Also  plotted  in  this  figure  is  the  opti¬ 
mum  probability  of  error  resulting  from  the  design  of  the  system  based  on  equation  (4.48) 
(solid  curve).  As  observed  from  the  figure,  the  difference  between  the  two  curves  can  be 
hardly  noticed.Therefore,  we  also  provide  numerical  results  in  Table  4.1.  Due  to  the  sym¬ 
metry  of  the  curve,  numerical  results  for  for  only  the  values  of  Ttg  £  0.5  are  presented. 
Note  that  the  thresholds  in  Table  4.1,  are  those  specified  by  (4.71). 
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Fig.  4.5.  The  optimum  fusion  rule  in  a  two-sensor  decentralized  detection  system. 
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Fig.  4.7.  The  optimum  fusion  rule  in  a  three-sensor  decentralized  detection  system. 
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Fig.  4.8.  Minimum  probability  of  error  for  an  M-level  quantization  two-sensor  decentral¬ 
ized  detection  system. 
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Fig.  4.9.  Variation  of  Minimum  probability  of  error  for  different  systems  in  terms  of  the 

signal  level  m  when  tcq  =  0.5. 
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Threshold  that  POE  obtained  from  Optimum  Threshold  Optimum 

Kq  minimizes  the  the  upper  bound  for  the  decentralized  MPOE 

upper  bound  minimization  approach  detection  system 


0.5 

0.2843 

0.1810 

0.2268 

0.1804 

0.6 

0.4461 

0.1726 

0.4047 

0.1723 

07 

0.6167 

0.1529 

0.5925 

0.1528 

0.8 

0.8185 

0.1209 

0.815 

0.1209 

0.9 

1.1177 

0.07353 

1.1375 

0.0735 

Table  4.1 


4.6  Summary 

In  this  chapter  the  design  and  perfcmance  of  minimum  average  cost  decentralized 
detection  system  was  consideredThe  analysis  was  based  on  the  expression  derived  in 
Chapter  2  for  the  minimum  average  cost  of  an  optimum  receiver  in  terms  of  the  Kolmog¬ 
orov  variational  distance.  Both  hard  decision  and  soft  decision  systems  were  considered. 
Performance  enhancement  when  additional  detectors  are  added  was  determined.  A  design 
approach  for  the  MAC  decentralized  detection  system  based  on  the  above  results  was  pre- 
sentedThis  computational  procedure  is  much  simpler  than  the  previously  available  meth¬ 
ods.  As  an  example.design  of  distributed  detection  systems  for  MPOE  criterion  was 
considered  in  detail.Its  performance  was  compared  to  the  suboptimum  systems  designed 
based  on  some  Ali-Silvey  distance  measures. The  performance  degradation  of  distributed 
detection  systems  relative  to  the  cen-  lized  system  was  also  determined.lt  was  also  shown 
that  the  performance  of  the  decentralized  system  approaches  the  performance  of  the  cen¬ 
tralized  system  very  quickly  as  a  function  of  the  number  of  quantization  levels. 
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CHAPTER  5 


PERFORMANCE  EVALUATION  OF  DISTRIBUTED 
BAYESIAN  DETECTION  STRUCTURES 

5.1  Introduction 

We  have  previously  discussed  the  computational  difficulties  associated  with  the  pro¬ 
cedures  for  the  design  of  optimum  decentralized  detection  systems.  Therefore,  the  study 
of  decentralized  detection  systems  has  been  limited  to  small  networks  and  very  few  topol¬ 
ogies.  A  computationally  simpler  approach  for  the  design  of  decentralized  Bayesian  detec¬ 
tion  systems  was  presented  in  Chapter  4.  This  approach  is  based  upon  the  alternative 
expression  for  the  minimum  average  cost  derived  in  Chapter  2.  The  main  objective  of  this 
chapter  is  to  apply  this  approach  to  the  design  and  study  of  four  decentralized  detection 
topologies.  It  is  demonstrated  by  means  of  illustrative  examples  that  relatively  large  net¬ 
works  can  be  handled  rather  easily.  For  these  systems,  we  show  that  the  design  of  the  opti¬ 
mum  system  can  be  reduced  to  the  optimization  of  a  single  function  of  a  certain  number  of 
variables  that  depend  upon  the  configuration  considered.  In  Section  5.2,  we  briefly  formu¬ 
late  the  problem.  In  Section  5.3,  we  revisit  the  parallel  fusion  network  with  n  local  detec¬ 
tors  discussed  in  Chapter  4.  Here  we  deal  with  the  more  general  situation  where  the 
observations  received  by  the  local  detectors  are  not  necessarily  identical  and  where  the  lo¬ 
cal  thresholds  are  not  assumed  to  be  identical.  Under  these  conditions,  we  show  that  the 
design  of  the  optimum  system  reduces  to  the  optimization  of  a  single  function  of  n  vari¬ 
ables.  This  optimization  is  performed  only  once  with  no  need  to  search  over  t  II  the  possi¬ 
ble  fusion  rules,  i.e.,  the  optimization  procedure  is  not  exhuastive.  In  Section  5.4,  we 
consider  a  variation  of  the  parallel  system  where  we  allow  the  fusion  center  to  make  its 
judgement  based  on  the  received  local  decisions  as  well  as  its  directly  received  observa¬ 
tion  (side  information).  While  for  the  parallel  fusion  system  there,  the  number  of  possible 
fusion  rules  grows  exponentially  with  n  (see  Table  1.1),  there  are  2n  decision  thresholds 
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for  the  system  with  side  information.  These  decision  thresholds  span  the  entire  observa¬ 
tion  space  of  the  local  observations.  For  this  structure  we  show  again,  that  the  design  of 
the  optimal  system  reduces  to  the  optimization  of  a  single  function  of  n  variables.  In  Sec¬ 
tion  5.5,  we  consider  the  design  of  a  hierarchical  system  with  2n  local  detectors  and  n  re¬ 
gional  decision  makers.  The  regional  decision  makers  are  assumed  to  make  their  decisions 
based  on  the  decisions  received  from  the  local  decision  makers  and  also  on  their  own  ob¬ 
servations,  i.e.,  side  information.  The  design  of  the  optimum  system  for  this  structure  re¬ 
duces  to  the  optimization  of  a  function  of  3n  variables.  In  Section  5.6,  we  discuss  the 
design  of  the  hierarchical  system  in  which  the  regional  decision  makers  do  not  have  obser¬ 
vations  of  their  own.  In  Section  5.7  we  present  several  examples  along  with  some  numeri¬ 
cal  results.  Section  5.8  contains  a  discussion  and  some  concluding  remarks. 

5.2  Problem  Formulation 

In  Chapter  4,  we  developed  a  computationally  simple  approach  to  the  design  of 
decentralized  detection  systems.  This  procedure  is  based  upon  an  alternative 
representation  of  the  minimum  average  cost  in  terms  of  a  modified  form  of  the 
Kolmogorov  variational  distance.  The  procedure  was  applied  to  the  design  of  the  parallel 
fusion  network  when  the  incoming  observations  were  assumed  independent  and 
identically  distributed  and  the  local  thresholds  were  assumed  to  be  identical.  The  cases  of 
hard  decisions  and  soft  decisions  were  analyzed.  In  this  Chapter  we  apply  the  design 
procedure  to  the  design  of  a  number  of  decentralized  detection  structures  including  the 
parallel  fusion  network.  These  structures  are  used  for  the  binary  hypothesis  testing 
problem.  The  null  hypothesis  Hq  with  a  priori  probability  Kq  is  tested  against  the 
alternative  hypothesis  Hj  with  a  priori  probability  7tj.  The  criterion  we  adopt  is  the 
minimization  of  the  system  average  cost.  We  consider  the  case  when  the  local  as  well  as 
the  regional  decisions  are  either  0  (corresponding  to  Hq)  or  1  (corresponding  to  Hj),  i.e., 
we  only  treat  the  case  of  hard  decisions.  Here,  we  assume  that  the  incoming  observations 
are  independent  but  not  necessarily  identical .  Let  Qj ;  i,  j  =  0,  1 ,  be  the  overall  cost  of 
deciding  ilj  when  Hj  is  true.  Since  the  final  decision  is  made  at  the  fusion  center,  these 
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costs  are  applicable  there,  and  the  fusion  center  can  be  looked  upon  as  a  minimum  average 
cost  receiver.  Therefore,  the  minimum  average  cost  at  the  fusion  center  can  be  represented 
as  given  in  (2.21).  Now  we  apply  our  computationally  simple  approach  to  the  design  of 
several  network  topologies. 

5.3  Parallel  Fusion  Network 


Let  us  consider  the  system  Sj  shown  in  Fig.  1.1  consisting  of  n  local  detectors  and  a 
global  decision  maker.  The  system  receives  n  observations  Xi,...,Xn  in  which  X,  denotes 
the  observation  received  by  the  local  detector  LDj,  1  <  i  <  n.  We  assume  that  these  obser¬ 
vations  are  independent  with  conditional  pdf’s  po(xi)  and  pi(xj)  under  hypotheses  H0  and 
Hi  respectively.  Due  to  the  bandwidth  constraints  on  the  channels  linking  the  local  detec¬ 
tors  to  the  global  decision  maker,  the  local  detector  LDj  compresses  its  raw  observation  Xj 
to  a  single  hard  decision,  Zj,  indicating  whether  Hq  or  Hi  is  true  and  transmits  it  to  the  fu¬ 
sion  center.  The  design  procedure  employed  here  can  be  easily  extended  to  the  design  of 
decentralized  detection  systems  employing  soft  decisions,  but  this  case  will  not  be  consid¬ 
ered  here.  We  assume  that  the  decision  made  by  the  local  detector  LD]  is  independent  of 
the  decisions  made  by  the  other  local  detectors.  Due  to  the  independence  assumption,  in 
the  optimum  system,  each  local  detector  LDj  performs  a  local  likelihood  ratio  test  with  re¬ 
spect  to  some  local  threshold  Xj.  That  is,  the  decisions  Zj  are  made  based  on  the  following 
rule 


Zi= 


{ 


1  if  PliXiVpoiXi)  >Tj 


i=l,...,n  (5.1) 


0  otherwise 

where  Zj  =  1  means  that  Hi  has  been  declared  true  and  Zj  =  0  means  that  Hq  has  been  de¬ 
clared  true  by  the  local  detector  LDj.  The  decision  rule  (5.1)  thus  characterizes  each  local 
detector  LDj  by  a  local  probability  of  detection  PDL ,•  and  a  local  probability  of  false  alarm 
PFL^  Each  element  of  the  local  decision  vector  U  =  [zi...Zj,]  is  a  Bernoulli  random  vari¬ 
able  having  the  following  density  functions  under  the  two  hypothesis 


/o(z  i)=PFLjZi(  1  -PFLi) 1  (5.2) 


102 


f}(zi)=PDL*i(  1  -PDLi) 1  ~zi 


(5.3) 


It  is  to  be  noted  that  unlike  the  case  considered  in  Chapter  4  where  the  decisions  zj,  Z2, 

Zj,  were  independent  and  identically  distributed,  the  decisions  zj,  Z2, zn  here  are  inde¬ 
pendent  but  are  not  assumed  to  be  identically  distributed.  This  is  a  result  of  our  earlier  as¬ 
sumption  that  the  observations  Xj,  X2, ....  Xn  are  not  necessarily  identically  distributed. 
Using  the  assumption  of  independence  between  the  local  decisions  Zj,  the  decision  vector 
U  has  the  following  density  function  under  the  two  hypotheses 

n 

ftm  =  =  °.i  (5.4> 

i=  1 

These  decisions  are  sent  over  bandlimited  channels  to  the  global  decision  maker.  The  glo¬ 
bal  decision  uq  is  obtained  by  the  fusion  center  based  on  the  vector  U  by  performing  the 
likelihood  ratio  test  (4.2)  with  A (U)  =  f\(U)  / /q(C/).  Using  (2.21),wC  can  find  the  MAC 
of  the  system  as 

*  =  *0-5£|C,/,(W-C0/0(f/)|  (5.5) 

We  should  point  out  that  of  all  the  possible  fusion  rules,  (4.2)  specifies  the  fusion  rule  that 
achieves  the  smallest  MAC  for  a  given  set  of  local  decision  rules.  The  cost  corresponding 
to  this  (best)  fusion  rule  is  specified  by  equation  (5.5).We  also  emphasize  that  an  explicit 
knowledge  of  the  fusion  rule  is  not  required  to  determine  the  MAC  given  in  (5.5).  This 
point  was  illustrated  by  means  of  an  example  in  Section  4.4. 

While  (5.5)  determines  the  MAC  for  given  a  priori  probabilities,  cost  assignments, 
and  local  decision  rules,  it  can  also  be  used  for  designing  the  optimum  system.  In  order  to 
use  (5.5)  for  this  objective  we  let  x  =  [Xi...xn]  denote  the  vector  whose  elements  are  the  lo¬ 
cal  thresholds,  where  Xj  is  the  threshold  of  local  detector  LDj  as  defined  in  (5.1).The  local 
detection  probability  PDLi  and  the  local  false  alarm  probability  PFLt  of  local  detector  LD, 
are  functions  of  Xj.  We  stress  this  by  expressing  PDL,  as  PDL,(Xj)  and  PFLl  as  PFL ,(Xj). 
Therefore,  we  can  express  the  conditional  distributions  given  by  (5.2)  and  (5.3)  of  the  lo- 
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(5.6) 


cal  decisions  in  terms  of  x,  as 

/0(Zi(Xi))=  [PFLfcdfi  [l-PFLffll^i 


/l(z i(xi))=[/»DLi(xi)]zi  [l-PDLidi)]1-2!  (5.7) 

The  conditional  densities  of  the  decision  vector  U  correspondingly  can  be  expressed  in 
terms  of  x  as 

n 

fj(U(x))  =  n/,(^»,y  =  O’1  (5.8) 

i=  1 

For  a  given  vector  x  of  local  thresholds,the  observation  space  of  U  is  optimally  par¬ 
titioned  using  (4.2)  with  AQJ)  expressed  in  terms  of  the  densities  given  in  (5.8)  as 

Aom)) 

a  u/(D)  =  foa/(x))  (5-9) 

As  pointed  out  in  the  second  remark  in  Section  4.2,  the  decision  rule  (4.2)  with  A(x)  as 
given  in  (5.9)  specifies  the  fusion  rule  with  the  smallest  MAC  for  the  given  vector  x.  The 
MAC  corresponding  to  this  fusion  rule  with  n  incoming  decisions  expressed  in  terms  of  x 
is 


*„(J)  =^o-0-5XlCi/i^^>-C(/o^(-T))|  (5-10) 

u 

where  the  summation  is  taken  over  all  the  possible  values  of  U.  Equation  (5.10)  is  a  func¬ 
tion  of  n  variables,  namely,  the  n  local  thresholds  Xj, ...,  xn.  For  a  given  value  of  x.  Equa¬ 
tion  (5.10)  determines  the  cost  corresponding  to  the  best  fusion  rule  among  all  the  possible 
fusion  rules.  Therefore,  to  optimally  design  the  overall  system,  Rn(x)  should  be  minimized 
with  respect  to  x.  The  resulting  fusion  rule  can  be  determined  from  (5.2)  (see  also  Appen¬ 
dix  A). 

If  the  observations  Xj, ....  Xn  are  independent  and  identically  distributed  under  both 
hypotheses,  then  the  receiver  operating  characteristics  of  the  n  local  detectors  are  identi- 
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cal.  That  is,  if  a!  and  are  the  false  alarm  and  detection  probabilities  that  are  achieved  at 
threshold  x  for  one  of  the  local  detectors,  then  the  false  alarm  and  detection  probabilities 
of  all  the  other  detectors  at  threshold  x  will  also  be  aj  and  pj.  If  we  assume  that  these  local 
detectors  when  operating  in  a  decentralized  system  have  identical  thresholds,  then  we 
have  PDLi  =  PDL  and  PFL,  =  PFL  for  1  <  i  <  n.  Eventhough  this  assumption  is  intuitively 
appealing,  a  number  of  counterexamples  have  been  reported  in  the  literature  [20,58] 
whereby  the  overall  system  cost  is  minimized  by  nonidentical  local  decision  rules  ev¬ 
enthough  the  local  detectors  are  identical.  It  has  been  observed  in  [24]  that  for  decentral¬ 
ized  Beyesian  detection  systems,  the  identical  local  decision  rules  assumption  often  results 
in  little  or  no  loss  of  optimality.  The  system  design  procedure  greatly  simplifies  for  the 
identical  detector  case.  In  this  case  the  vector  x  reduces  to  a  scalar  x  and  the  n-variable  de¬ 
sign  equation  (5.10),  reduces  to  equation  (4.47)  which  is  a  function  of  one  variable  only. 
In  Example  5.1,  we  will  use  both  equations  (5.10)  and(5.47)  to  design  a  system  consisting 
of  six  local  detectors  with  identical  observation  statistics. 


5.4  Parallel  Fusion  Network  with  Side  Information 


In  this  section  we  consider  the  distributed  detection  system  S2  shown  in  Fig.  1.2. 
This  system  is  different  from  the  one  treated  in  the  previous  subsection  in  that  the  global 
decision  maker  receives  a  local  observation  of  its  own  (or  side  information)  in  addition  to 
the  local  decisions  zj.  The  observation  vector  based  on  which  the  fusion  center  makes  the 
final  decision  is  the  augmented  vector  [UXqI  of  the  local  decisions  Zj  and  the  observation 
Xq  at  the  fusion  center.  The  global  decision  is  made  according  to  the  rule 


where 


rm 

A(*o)  - 

fodD 


Hi 

> 

< 

H0 


AU0) 


P 1  Up) 
Pq  (Xq) 


(5.11) 


105 


and  ti  is  the  global  threshold  defined  in  equation  (4.2).  An  equivalent  test  can  be  obtained 
in  terms  of  the  local  observation  Xq  as 

Hi 

>  foOD 

A(x0)  T1  -  =TlQ,Q=l,...,2n  (5.12) 

<  AOD 

Ho 


By  using  the  expressions  given  in  (5.4)  for fo(U)  and  f\(U)  and  taking  the  logarithm  of 
both  sides  of  (5.12)  we  get  the  following  (see  Appendix  A  for  details) 


logri 


foill) 

ML 1) 


"  1  -  PFLi 

=  logT)  +  2,  log 
i=  1 


+ 


X  z<loS 

i  =  1 


PFLi  ( 1  -  PDL ,) 
PDLi  ( 1  -  PFL) 


(5.13) 


Note  that  the  equivalent  test  in  terms  of  the  logarithm  specified  by  (5.12)  and  (5.13)  is  an 
extension  of  the  optimum  data  fusion  algorithm  developed  in  [91.  Here,  the  locally  re¬ 
ceived  observation  Xq  has  been  taken  into  account.  As  can  be  seen  from  this  new  equation 
(giving  the  test),  there  are  2n  different  decision  thresholds  to  be  employed  at  the  global  de¬ 
cision  maker.  Each  threshold  Tig,  Q  =  1, ...,  2n  corresponds  to  a  particular  sequence  of  the 
2n  possible  values  that  the  decision  vector  U  takes.  Using  (2.20)  we  can  determine  the 
minimum  cost  of  the  system  as 

R  =  R0-\j'Z\Cifi(y'*Pi(xo)  -Cof0(U)p0(x0)\dx0  (5.14) 

^  u 


where  the  integration  is  performed  over  all  values  of  Xq.  When  the  observations  Xi„.Xn 
are  independent  and  identically  distributed  under  both  hypotheses,  the  receiver  operating 
characteristics  of  the  n  local  detectors  become  identical.  If  we  assume,  as  we  did  in  the 
previous  subsection,  that  the  local  detectors  use  identical  thresholds,  then  the  equivalent 
thresholds  given  in  (5.13)  simplify  to 


logT) 


/o  00 

MU) 


=  logri  ( 


1  ~PFL  n  PFL  ( 1  -  PDL) 
1  -PDL}  +  °gPDL(\-PFL) 


(5.15) 


Equation  (5.15)  indicates  that  there  are  (n+1)  decision  thresholds  employed  at  the  global 
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decision  maker.  If  we  let  K  be  the  number  of  local  thresholds  that  decide  in  favor  of  Hj, 
then  K  has  the  distributions  given  by  (4.34)  and  (4.35).  Using  (4.34)  and  (4.35)  the  MAC 
in  (5.14)  can  be  expressed  as 


As  before,  we  let  x  =  [x  j  ...x„]  be  the  vector  of  local  thresholds.  We  express  PDLl  as 
PDLi(x{)  and  PFLi  as  PFL^x j).  For  any  setting  of  the  local  thresholds  x,  the  optimum  par¬ 
titioning  of  the  observation  space  of  Xq  is  made  according  to  the  decision  rule  (5.12).  The 
decision  thresholds  riq  expressed  explicitly  in  terms  of  x  are 

n/o  <*,(*,•>> 

ne(J)=’liv -  (5.17) 

n/,u.A» 

i  =  1 

where  the  density  functions  in  the  numerator  and  the  denominator  are  those  given  in 
equations  (5.6)  and  (5.7).  The  resulting  system  MAC  from  (5.14)  can  be  expressed  as  a 
function  of  x  as 

*„(D  =  *o-5J<Xl<Vi (tfdDPiUo)  -<Vo(I(<J>>P0<*0>|)‘fro 

(5.18) 

If  the  local  threshold  vector  x  is  changed  to  x"  it  results  in  a  different  set  of  2n  decision 
thresholds  at  the  fusion  center  and,  consequendy,  a  different  value  of  the  global  MAC, 
Rn(x").  Our  goal  is  to  obtain  the  best  global  MAC.  Therefore,  Rn(x)  should  be  minimized 
with  respect  to  the  local  threshold  vector  x.  The  resulting  system,  i.e.,  the  resulting  local 
threshold  vector  x  and  the  corresponding  2n  decision  thresholds  specified  by  (5.12),  is  op¬ 
timum.  It  should  be  emphasized  that  equation  (5.18)  is  a  function  of  n  variables,  namely, 
the  n  local  thresholds.  The  explicit  dependence  of  the  cost  on  the  2n  decision  thresholds  at 
the  fusion  center  has  been  avoided. 
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The  design  equation  (5.18)  simplifies  tremendously  for  the  identical  thresholds  case. 
In  this  case  the  vector  x  reduces  to  a  scalar  threshold  x.  The  decision  thresholds  specified 
by  (5.17)  become 

PFL*(x)[l-PFL(x)]n'* 

T[k=T\ -  ,k  =  0 . n  (5.19) 

PDLk(x)[l-PDL(x)]n-k 

The  system  MAC  in  terms  of  x,  Rn(x),  is  obtained  by  explicitly  expressing  PFL  as  PFL(x ) 
and  PDL  as  PDL(x)  in  equation  (5.16).  In  this  case  the  cost  function  is  a  function  of  one 
variable,  namely,  the  common  local  threshold.  The  global  minimum  cost  is  obtained  by 
minimizing  Rn(x)  with  respect  to  the  local  threshold  x.  The  resulting  local  threshold  along 
with  the  corresponding  (n+1)  decision  thresholds  determined  by  (5.19)  specify  the  opti¬ 
mum  system.  An  example  will  be  presented  in  Section  5.7. 

5.5  Hierarchical  System  with  Side  Information  at  the  Regional  Detectors 

In  this  subsection  we  consider  the  hierarchical  decentralized  detection  system  S3 
shown  in  Fig.  1.3.  The  system  consists  of  2n  local  decision  makers,  n  regional  decision 
makers  (RD’s)  and  a  global  decision  maker.  Local  detectors  LD2j.j  and  LD2i  process  their 
locally  received  observations  X2j_]  and  and  forward  their  decisions  Z2j_]  and  z2j  to  an 
intermediate  regional  decision  maker  RD,,  i  =  1, ...,  n.  In  the  hierarchical  system  consid¬ 
ered  in  this  chapter,  the  local  decisions  of  only  two  detectors  are  combined  at  the  regional 
decision  makers.  The  results  can  be  generalized  to  the  case  of  more  than  two  local  detec¬ 
tors  per  regional  decision  maker.  However,  for  simplicity  in  presentation,  we  consider  the 
case  of  two  local  detectors  only.  The  regional  detector  RDj  combines  the  two  local  deci¬ 
sions  along  with  its  directly  received  observation  Yj  to  make  the  regional  decision  uj.  The 
observations  Yj,  j  =  1, ...,  n  are  assumed  to  be  independent  with  pdf’s Poiyj)  and p\{yp  un¬ 
der  the  hypotheses  Hq  and  Hj  respectively.  The  decision  vector  U.=  [uj...un]  is  used  by  the 
global  decision  maker  to  make  the  final  decision  uq.  The  local  decisions  zv  i  =  1,2, ...,  2n 
are  made  based  on  the  decision  rule  (5.1)  and  hence,  each  local  detector  is  characterized 
by  a  local  probability  of  detection  PDL,  and  a  local  probability  of  false  alarm  PFL,.  The  fi- 
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nal  decision  is  made  based  on  the  decision  rule  (4.2)  with  A(U)  to  be  specified  later.  Let 
T|j,  i  =  1, ...,  n  be  the  unknown  threshold  of  the  regional  detector  RDj.  The  regional  deci¬ 
sion  Uj  is  made  according  to  the  test 


P\(yMz2i-\Mz2i)  > 

Po(yiMz2i-Mz2i)  < 

Ho 


"Hi  ,  i  =  1,2  ,...,  n 


(5.20) 


where  fi(z,);  j  =  0,  1,  i  =  1,  2, ...,  2n  are  given  by  equations  (5.2)  and  (5.3).  The  test  (5.20) 
can  also  be  expressed  in  terms  of  the  regional  observation  Yj  as  follows 


P\(y0 


Po(yt) 


Mz2i-}Yo(z2i) 

*1; - -  r\ikJ  ,k,i  =  o,  l 

fl(z2i-lVl(z2i) 


(5.21) 


where  k  is  equal  to  Z2i.\,  the  decision  of  local  detector  LD2i-i>  and  /  is  equal  to  Z2,.  the  de¬ 
cision  of  local  detector  LD2j.  Depending  on  the  values  that  z2,_i  and  z2j  take,  the  regional 
decision  maker  RDj  employs  one  of  the  following  thresholds 


(\-PFL2i_x)(l-PFL2i) 
^,0°  ^'d -PDL2i_x)  (1  -PDL2i) 


(5.22-a) 


_  (1  -PFL2l_x)PFL2i 

^‘°i  ( 1  -  PDL2i  _  ^  PDL2i 


(5.22-b) 


_  PFL2l_x(\-PFL2l) 
n*i°  ^iPDL2l_l  (1  -PDL2i) 


(5.22-c) 
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(5.22-d) 


PFL2i_xPFL2i 
11/11  11 iPDL2i_xPDL2i 

Let  PDRtfftju)  and  PFR^r \ikl)  denote  detection  and  false  alann  probabilities  of  regional 
decision  maker  RDj  for  a  given  threshold  ,  i.e.,  when  z2i-l  =  k  an^  z2i  =  Tbese  prob¬ 
abilities  are  given  as 

PDR^ki)  =  P  (u i  =1  lz2M  =k,z2l  =1  Mi)  (5.23-a) 

PFFi^Ud)  =  P  (u i  =1  Iz2i-i  =k,z2l  =/  ,Ho )  (5.  23-b) 

Then  using  the  theorem  of  total  probability, we  can  find  the  unconditional  detection  and 
false  alarm  probabilities  as 

PD/?,  =  X  X  P(Z2M  =  k ,  Z2i  =  l  IHj)  PDRfaiu)  (5.24-a) 
k  l 


PFRi  =  X  Xp( z2i.i  =  k,  z2 i  =  /  IHp)  PFR^m)  (5.24-b) 
k  l 

Expanding  over  k  and  /  we  can  express  PDP,  and  PFP,  as 

PDRi  =  (/ -PDL2i.1)(l  -PDL2i)PDRj(r\i0Q) +(1-P DL2i.i )P DL2i  PD/?,( r\i01) 

+PDL2i.1a-PDL2i)PDRl(r[iI0)+PDL2l.]PDL2iPDRi(T\iI1)  (5.25-a) 
PFRi  =  {1-P FL2i.j){l -P FL2i)P 'FRfi\i0o)+(l -P F L2i.j )PF L2i  PFR^i01) 

+PFL2l_1(l-PFL2l)PFRl(T\il0)+PFL2i,]PFL2lPFRl(T}l]i)  (5.25-b) 
Each  regional  detector  RD,  is  characterized  by  a  detection  probability  PD/?,  and  a  false 
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alarm  probability  PFRj.  The  density  functions  of  the  regional  decision  Uj  under  Hq  and  Hj 
are  given  by  the  following  Bernoulli  distributions 

f0(Ui)  =  PFRfif  1  -PFR0 1 (5.26-b) 

fifty)  =  PDRiui(  1  -PDRj) 1  ‘“i  (5.26-a) 

for  u;  =  0, 1,  i  =  1 . n.  Using  the  independence  assumption  between  the  regional  deci¬ 

sions,  we  obtain  the  following  density  functions  for  the  decision  vector 

n 

fj(U)  =  Ylfj(u,)J=  <U  (5.27) 

i  =  1 

The  system  MAC  is  computed  using  (5.5)  with /qCLD  and/j(f/)  as  given  in  (5.27).  Let  x  = 
[Xl...t2n]  be  the  vector  of  local  thresholds  and  n=  fni-Tln]  be  the  vector  of  regional 
thresholds.  The  design  of  the  optimum  system  calls  for  the  determination  of  T,  n,  and  the 

fusion  rule  that  minimize  the  global  cost.  For  a  given  local  threshold  vy  j  =  1, 2 . 2n,  the 

local  detection  and  false  alarm  probabilities  PDLj  and  PFL,  are  functions  of  tj  as  discussed 
in  the  previous  sections.  For  a  given  regional  threshold  T|j,  i  =  1, ...,  n,  the  conditional  re¬ 
gional  thresholds  given  in  (5.22)  are  functions  of  the  local  thresholds  i2j_i  and  as  evi¬ 
dent  from  the  the  dependence  of  the  thresholds  in  (5.22)  on  the  local  detection  and  false 
alarm  probabilities.  These  thresholds  are  also  functions  of  rij  as  can  be  seen  from  (5.21). 
Consequently,the  unconditional  regional  detection  and  false  alarm  probabilities  given  in 
(5.25)  are  functions  of  the  variables  t2i,  and  T|j.  More  explicitly,  we  write  PDRi  as 
/>£>/?, (T2j_i,T2i,T|j)  and  PFRi  as  PFRi(X2 i-i.'c2i,Tli)-  ^  terms  of  the  system  variables,  the  sys¬ 
tem  MAC  in  (5.5)  becomes 


L  u 


n  n 

C1  n/l  (Ui  K’  hi- V  X2.)  )  -  CoYlfo  (“x  K’  hi-V  X2.)  ) 

i  =  1  i  =  1 

(5.28) 


Equation  (5.28)  is  a  function  of  3n  variables.  In  order  to  obtain  the  optimum  system,  equa- 


111 


tion  (5.28)  has  to  be  minimized  with  respect  to  these  variables.  The  fusion  rule  can  be  ob¬ 
tained  from  (4.2)  once  the  optimum  local  and  regional  thresholds  are  determined.  With  the 
identical  threshold  assumption  the  design  equation  (5.28)  becomes  a  function  of  two  vari¬ 
ables,  namely,  the  common  local  threshold  x  and  the  common  regional  threshold  tj. 

It  should  be  pointed  out  that  the  performance  of  this  system  is  expected  to  be  inferior 
to  those  in  the  preceding  sections  for  the  same  number  of  observations.  The  reason  for  this 
is  the  extra  data  compression  at  the  regional  level,  so  that  the  information  available  to  the 
fusion  center  is  less  than  before.  This  will  be  illustrated  in  Section  5.7  where  several  nu¬ 
merical  examples  are  considered. 

5.6  Hierarchical  Decentralized  Detection  System 

In  this  subsection  we  consider  the  hierarchical  detection  system  S4  shown  in  Fig. 
1.4.  The  difference  between  this  system  and  the  one  treated  in  the  previous  section  is  that 
the  regional  decision  makers  have  no  observations  of  their  own.  Therefore,  they  have  to 
make  their  decisions  solely  on  the  basis  of  the  local  decisions  they  receive.  The  local  deci¬ 
sions  and  the  final  decisions  are  still  made  as  in  Section  5.5,  i.e.,  the  local  decisions  are 
made  on  the  basis  of  the  local  observations  and  the  global  decision  is  made  on  the  basis  of 
the  decisions  received  from  the  regional  decision  makers.  However,  due  to  the  unavail¬ 
ability  of  observations  at  the  regional  detectors,  special  attention  has  to  be  paid  to  decision 
making  at  the  regional  decision  makers.  In  the  rest  of  this  section,  we  modify  the  optimi¬ 
zation  procedure  of  Section  5.5  to  take  into  account  the  absence  of  side  information  at  the 
regional  detectors. 

Let  be  the  threshold  used  by  regional  decision  maker  RDj.  The  regional  decision 
Uj  is  made  based  on  the  test 

H, 

/l(z2i-l)/l(z2i)  > 

^(*2i-1^2i)  =  -  Hi  (5.29) 

/o(z2i-l)/o(z2i)  < 

Ho 
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Substituting  the  density  functions  for  z ^  given  by  (5.2;  and  (5.3)  into  (5.29)  we  get 


H, 

(PDL2i.i)z2i-l  ( 1  -PDLft. ! ) 1  *z2i- 1  {l-PDL^^lx  > 


(PFL 2i-i)z2i-l  ( 1  -PFLii_  i ) 1  'z2i- 1  (PFL^fli  (1  -PFL^^li  < 

Ho 


T)i  (5.30) 


Taking  the  logarithm  of  both  sides  of  (5.30)  and  arranging  terms  we  obtain  the  following 


test 


Hi 

> 

^2i-l  z2i-l  +^2i  z2i  Ci 
< 


Ho 


(5.31) 


where 


A 


2i-  1 


PDL2i_x(\-PFL2i .,) 
PFL2i_x  (1  - PDL2i_\) 


A 


2  i 


PDL2i  ( 1  -  PFL2i) 
PFL2i(\-PDL2i ) 


(1  -PFL2i_x)  ( 1  - PFL2i) 
‘  (1  -PDL2i_x)  (1  -PDL2i) 


The  random  variables  Z2i_i  and  Z2j  are  Bernoulli  random  variables  under  both  hypotheses, 
but  their  linear  combination  in  (5.31)  is  not.  Let 

Li  =  A2i- \z2i-  1  +  A2iz2  (5.32) 
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be  the  sufficient  statistic  at  the  regional  decision  maker  RDj.  The  regional  probabilities  of 
false  alarm  and  detection  are 


PFRi  =  P  (Ui  =  1IN0  Is  true)  =  P  {L,  >  Ct- 1//0)  (5.33-a) 


PDRt  =P  {u(=  l\H1  is  true)  =  P  (L{  >  Cl  I Hx }  (5.33-b) 

Now  we  show  how  equation  (5.28)  can  be  used  to  design  the  system.  In  Section  5.5,  a 
likelihood  ratio  test  at  the  regional  detectors  was  formulated  where  the  likelihood  ratio 
was  obtained  using  y;  as  the  observations  and  the  incoming  local  decisions  were  used  to 
modify  the  threshold  q,.  In  that  case,  the  receiver  operating  characteristic  at  the  regional 
decision  maker  was  continuous  and  all  values  of  rj  j  were  permissible.  Thus  it  was  possible 
to  express  the  regional  probabilities  of  detection  and  false  alarm  in  terms  of  the  regional 
threshold  q*  and  the  probabilities  of  detection  and  false  alarm  of  the  local  detectors  con¬ 
nected  to  this  regional  detector.  The  optimum  system  was  obtained  by  optimizing  the  cost 
in  (5.28)  with  respect  to  the  local  and  regional  thresholds.  All  of  these  thresholds  were  as¬ 
sumed  to  be  independent,  i.e.,  the  number  of  independent  variables  was  3n.  The  difficulty 
in  the  design  oi  the  system  of  Fig.  1.4  arises  due  to  the  fact  that  the  regional  thresholds  and 
the  local  thresholds  cannot  be  assumed  to  be  independent,  i.e.,  we  do  not  have  3n  indepen¬ 
dent  variables.  In  Appendix  C,  we  show  that  for  given  local  thresholds  i2i-i  and  t2j,  the  re¬ 
gional  threshold  q;  may  lie  in  one  of  five  possible  regions.  These  regions  are  determined  in 
terms  of  the  local  thresholds.  If  the  regional  threshold  T]j  lies  in  the  two  outside  regions,  it 
results  in  maximum  possible  values  of  system  cost.  Therefore,  these  two  regions  are  not 
desirable.  In  the  remaining  three  regions  the  system  cost  does  not  change  as  the  regional 
threshold  q,  is  varied  within  any  of  these  regions.  Thus  it  is  not  possible  to  express  the  re¬ 
gional  probabilities  of  detection  and  false  alarm  in  terms  of  the  regional  threshold  qj  ex¬ 
plicitly.  The  functional  relationship  between  X2i_i  and  t2j  and  q4  can  be  further  emphasized 
by  expressing  qj  as  qi(X2i.i,X2i).  The  optimization  procedure  of  Section  5.5  can  now  be 
used  except  that  we  have  only  two  independent  variables  x2i-i  and  x2j  for  each  regional  de¬ 
tector  RDj. 
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5.7.  Examples 


In  this  section  we  present  several  numerical  examples  that  illustrate  the  utility  of  the 
design  procedure  described  in  this  chapter.  Our  goals  in  presenting  these  examples  are 

1)  To  examine  the  effect  on  the  global  MAC  of  allowing  identical  local  detectors  to  use 
identical  local  decision  rules. 

2)  To  examine  the  effect  of  having  side  information  at  the  regional  and  global  decision 
makers. 

3)  To  compare  the  MPOE  of  the  various  systems  when  the  total  number  of  observation  re¬ 
ceived  by  each  system  is  six. 

In  these  examples  we  assume  that  under  hypothesis  Hq,  each  local  and  regional  observa- 
tion  is  a  Gaussian  random  variable  with  mean  zero  and  variance  a,  .  While  under  hypoth- 
esis  Hj,  each  observation  is  a  Gaussian  random  variable  with  mean  m  and  variance  a,  .  As 
discussed  in  Example  5.2,  for  the  Gaussian  hypothesis  testing  problem,  it  more  conve¬ 
nient  to  express  the  local  thresholds  in  terms  of  the  local  obcervat;ons  as  (see  equation 
(4.71)). 


(5.34) 


where  xx  represent  tl.e  local  thresh  >ids  defined  in  equation  (5.1).  Note  that  the  thresholds 
that  Dpear  in  the  tables  at  the  end  of  this  section  are  those  given  by  (5.34).  We  should 
mer  that  in  a’l  the  systems,  the  design  procedure  reduces  to  the  minimization  of  a  cost 
function.  In  performing  the  optimization  we  use  the  method  developed  by  Hooke  and 
Jeeve  [69]. 

Example  5.1 

The  objective  of  this  example  is  to  examine  whether  optimality  is  lost  by  assuming 
that  the  identical  local  detectors  in  systems  Sj  and  S2  are  using  the  same  threshold. 

Consider  the  system  Si  with  six  local  detectors  and  the  system  S2  with  five  local  de- 
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tectors  so  that  both  the  systems  have  an  identical  number  of  local  observations,  i.e.,  six  lo¬ 
cal  observations.  Let  Kq  =  0.5,  jij  =  2.5,  and  a  2  =  1  for  all  the  observations  in  the  two 
systems.  Using  different  cost  assignments  we  perform  the  design  of  the  systems  for  two 
cases.  In  the  first  one,  we  do  not  restrict  the  local  detectors  to  employ  an  identical  thresh¬ 
old,  i.e.,  in  obtaining  the  optimum  local  thresholds,  we  assume  that  each  one  of  the  local 
detectors  employs  a  different  threshold.  The  appropriate  design  equations  in  this  case  are 
(5.10)  for  system  Si  and  (6.18)  for  system  S2  which  are  functions  of  six  variables  and  five 
variables  respectively.  In  the  second  case,  we  assume  that  all  local  detectors  have  identical 
thresholds.  The  appropriate  design  equations  are  (4.47)  for  system  Si  and  (5.18)  for  sys¬ 
tem  S2.  For  both  systems,  the  thresholds  obtained  with  and  without  equal  threshold  restric¬ 
tion  were  identical  for  a  variety  of  cost  assignments.  In  other  words,  for  this  example 
optimality  is  not  lost  if  identical  threshold  assumption  is  made  when  observation  statistics 
at  the  sensors  are  identical.  The  values  of  the  optimum  local  thresholds  and  the  optimum 
costs  are  shown  in  Tables  5.1  and  5.2.  Also  shown  in  Table  5.1  are  the  optimum  fusion 
rules  for  system  These  fusion  rules  are  obtained  by  substituting  the  optimum  local 
thresholds  in  the  likelihood  ratio  function  (5.9)  and  using  (4.2)  (see  also  Appendix  A).  In 
Table  5.2,  we  have  also  included  the  cost  for  the  optimum  centralized  system,  i.e.,  where 
all  six  observations  are  processed  centrally.  The  performance  loss  due  to  decentralization 
is  evident  from  Table  5.2.  Also,  note  that  decentralized  detection  system  with  side  infor¬ 
mation  performs  better  than  the  system  without  side  information. 

Example  5.2 

In  this  example,  we  examine  the  effect  of  havind  one  noisy  local  detector  on  the  per¬ 
formance  of  the  decentralized  systems  Sj  and  S2. 

Consider  the  hypothesis  testing  problem  of  Example  5. 1  for  the  MPOE  criterion.  We 
assume  that  the  assumptions  and  conditions  of  that  example  are  still  valid  except  for  one 
of  the  local  detectors  where  we  assume  that  its  noise  variance  a2  is  a  variable.  For  Sj,  this 
means  that  for  five  of  the  local  detectors  the  observation  statistics  under  the  two  hypothe¬ 
ses  are  given  as 
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Cqo  C11  Cio  C0i  Optimum  local  threshold  Optimum  cost  Optimum  fusion  rule 


0 

0 

1 

1 

1.46 

6.059  xlO'3 

3  out  of  6 

0 

0 

1 

5 

1.30 

1.266  xlO'2 

3  out  of  6 

0 

0 

5 

1 

1.20 

1.266  xlO'2 

4  out  of  6 

1 

2 

4 

6 

1.43 

1.5208 

4  out  of  6 

0 

0 

30 

40 

1.01 

0.2107 

4  out  of  6 

1 

1 

2 

2 

1.037 

1.006 

4  out  of  6 

Table  5.1:  Results  of  Example  5.1  for  the  parallel  fusion  network  Sj 

Coo 

C11 

C10 

C01 

Optimum  local  threshold 

Optimum  cost 
for  system  S2 

Optimum  cost  for  the 
centralized  system 

0 

0 

1 

1 

1.24 

5.216  xlO'3 

1.0998  xlO'3 

0 

0 

1 

5 

1.15 

1.119  xlO'2 

2.3877  xlO' 

0 

0 

5 

1 

1.36 

1.1192  xlO'2 

2.3877  xlO* 

1 

2 

4 

6 

1.23 

1.518 

1.5038 

0 

0 

30 

40 

1.23 

0.18046 

0.03806 

1 

1 

2 

2 

1.247 

1.00522 

1.0011 

Table  5.2  :  Results  of  Example  5.1  for  the  parallel  network  S2  with  side  information 
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Hq  :  Xj  ~  N(0,1) 

Hi :  Xj  ~  N(2.5,l) ,  i  =  1,2,...,5. 

while  for  the  sixth  local  detector  the  observation  statistic  is  given  as 
Ho  :X6~  N^o2) 

H^Xg-Nd^o2) 

For  S2  ,  the  observation  statistics  under  the  two  hypotheses  for  four  of  the  local  detectors 
and  the  side  observation  are  given  as 

Hq  :  Xj  ~  N(0,1) 

H1:Xi~N(2.5,l),0  =  l,2, 

while  for  the  fifth  local  detector  the  observation  statistic  is  given  as 
Ho  :  X5  ~  mo2) 

H!  :  X5  ~  N(2.5,o2) 

The  objective  is  to  examine  the  performance  of  the  systems  as  a  function  of  the  noise  level 
o2of  one  of  the  detectors  when  the  rest  of  the  system  remains  fixed.  The  cost  of  each  sys¬ 
tem  is  obtained  by  minimizing  the  appropriate  system  cost  equation.  The  solid  curves  in 
Figures  5. 1  and  5.2  respectively  show  the  MPOE  for  systems  Si  and  S2  versus  o2.  The 
broken  line  in  Figure  5.1,  shows  the  MPOE  of  the  parallel  fusion  system  when  the  noisy 
detector  is  disregarded,  i.e.,  the  MPOE  when  system  makes  the  final  decision  based  on  the 
five  non-noisy  detectors.  Similarly,  the  broken  line  in  Figure  5.2  represents  the  MPOE  of 
the  parallel  fusion  system  with  side  information  when  the  final  decision  is  made  based  on 
four  local  detectors  (excluding  the  noisy  detector)  and  the  side  observation.  The  broken 
lines  in  Figures  5.1  and  5.2  are  constants  for  all  the  values  of  cr2  because  the  decision  of 
the  noisy  detector  is  ignored  while  making  the  decision. 

Example  5.3 

In  this  example  we  compare  the  optimum  performance  of  system  Si  with  n  local  de¬ 
tectors  to  the  optimum  performance  of  system  S2  with  (n-1)  local  detectors  for  various 
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values  of  n. 


Let  m  =  1  and  Oj2  =  1  for  all  the  observations  in  the  two  systems.  Also  let  Cqo  =  Cjj 
=  0  and  C10  =  Qu  =  1.  First,  let  us  compare  the  performance  of  system  Sj  to  the  perfor¬ 
mance  of  system  S2  for  the  case  when  n  =  2  (see  Example  4.2).  Note  that  when  n  =  2,  the 
system  S2  reduces  to  the  two-stage  serial  system.  The  comparison  is  made  for  all  values  of 
7to ;  0  <  Hq  <  1.  The  optimum  minimum  probability  of  error  (MPOE)  of  system  Si  and  S2 
as  a  function  of  it q  is  shown  in  Fig.  5.3.  It  is  clear  from  this  figure  that  S2  performs  better 
than  Sj  for  all  values  of  jcq  .  These  results  are  consistent  with  the  results  obtained  in  [21, 
23].  In  Figure  5.3,  we  also  present  the  performance  of  the  optimum  centralized  system.  At 
the  point  7Cq  =  0.5,  we  see  that  the  MPOE  of  system  Sj  is  12.9  %  more  than  that  of  the 
centralized  system.  For  system  S2  the  corresponding  percentage  is  7.4  %  only.  Next,  we 
compare  the  performance  of  systems  Sj  and  S2  as  a  function  of  n  for  the  case  when  itQ  = 
0.5.  Based  on  the  observation  of  the  results  of  Example  5.1,  here  we  design  both  systems 
under  the  identical  thresholds  assumption  without  loss  of  optimality.  For  each  value  of  n, 
the  systems  Sj  and  S2  are  designed  using  (4.48)  and  (5.16)  respectively.  The  optimum 
thresholds  and  the  optimum  MPOE  are  shown  in  Table  5.3  along  with  the  MPOE  of  the 
centralized  system  with  n  observations.  As  can  be  seen  from  Table  5.3,  the  MPOE  of  the 
parallel  system  with  2  local  detectors  is  5.2  %  higher  than  the  MPOE  of  the  corresponding 
serial  system.  For  other  values  of  n  this  percentage  is  smaller.  As  expected,  for  large  n 
both  systems  have  essentially  the  same  performance.  As  n  gets  larger,  the  percentage  by 
which  the  MPOE  of  both  systems  are  higher  than  that  of  the  centralized  system  becomes 
larger.  For  the  parallel  system  with  n  =  2,  this  percentage  as  mentioned  above  is  12.9  %, 
while  for  the  case  with  n  =  10  it  is  73.6  %.  Thus  the  need  for  having  soft  decisions  rather 
than  hard  decisions  becomes  more  apparent  for  larger  n  if  the  overall  MPOE  is  to  be  with¬ 
in  some  reasonable  bounds  of  the  MPOE  of  the  centralized  system.  This  requires  the  trans¬ 
mission  of  more  bits  of  information  to  the  fusion  center.  This  problem  of  soft  decisions 
was  treated  in  Chapter  4. 
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Fig.  5.3.  Performance  comparison  for  the  parallel  fusion  network  Sj  and  the  serial  net¬ 
work  S2  for  n  =  2. 


122 


n 

Si  with  n  local  detectors 

S2  with  (n-1)  local  detectors 

Centralized 

Threshold 

MPOE 

Threshold 

MPOE 

MPOE 

2 

1 

0.271 

0.5 

0.2575 

0.2397 

3 

0.5 

0.2268 

0.5 

0.22244 

0.1932 

4 

0.782 

0.2014 

0.5 

0.19397 

0.1586 

5 

0.5 

0.1745 

0.5 

0.17078 

0.13176 

6 

0.695 

0.156 

0.5 

0.15108 

0.11032 

7 

0.5 

0.1373 

0.5 

0.13434 

0.09293 

8 

0.649 

0.1234 

0.5 

0.1198 

0.07864 

9 

0.5 

0.1095 

0.5 

0.10718 

0.0668 

10 

0.62 

0.09881 

0.5 

0.09608 

0.05692 

11 

0.5 

0.08824 

0.5 

0.08634 

0.04862 

12 

0.60 

0.07979 

0.5 

0.07769 

0.04163 

13 

0.5 

0.07157 

0.5 

0.07003 

0.03571 

14 

0.5875 

0.06484 

0.5 

0.06319 

0.03068 

15 

0.5 

0.05835 

0.5 

0.05713 

0.0264 

Table  5.3  :  Results  of  Example  5.3  for  the  comparison  of  the  MPOE  of  systems  S] 
and  S2  for  various  values  of  the  number  of  local  detectors  n. 
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Example  5.4 

Consider  the  hierarchical  system  S3  consisting  of  four  local  detectors  and  two  region¬ 
al  detectors.  We  compare  the  MPOE  of  this  system  to  the  MPOE  of  the  hierarchical  sys¬ 
tem  S4  with  six  local  detectors  and  three  regional  detectors  such  that  the  number  of 
observations  in  both  systems  is  six.  Let  Kq  -  0.5,  fij  =  2.5,  and  <5\  -  1  for  all  the  observa¬ 
tions.  The  optimum  system  is  obtained  by  minimizing  the  cost  in  (5.28).  The  minimum 
cost  (MPOE)  for  system  S3  is  1.0809xl0*2.  The  local  thresholds  were  found  to  be  identi¬ 
cal  and  equal  to  1.47.  The  regional  thresholds  were  also  identical  and  we  obtained  131  =  132 
=  10.09.  For  system  S4,  the  minimum  cost  was  found  to  be  1.1874xl0*2.  It  turns  out  that 
there  are  two  possible  solutions  that  achieve  this  MPOE.  In  Table  5.4,  the  values  of  the  op¬ 
timum  local  thresholds  are  given  in  addition  to  the  fusion  rules  employed  at  the  regional 
detectors.  Also  given  in  this  table  are  the  permissible  regions  of  the  regional  threshold  at 
the  regional  detectors.  These  regions  are  derived  in  Appendix  C.  It  is  to  be  noted  from  this 
table,  that  the  optimum  solutions  are  achieved  with  nonidentical  local  thresholds.  Howev¬ 
er,  the  local  thresholds  of  the  detectors  linked  to  the  same  regional  detector  were  found  to 
be  identical.  For  the  sake  of  completeness,  we  summarize  below  the  performance  of  the 
systems  Sj,  S2,  S3,  and  S4  for  the  MPOE  criterion  assuming  that  p.j  =  2.5  and  a  2  =  1  for 
all  of  the  observation 

MPOE  of  Si  with  6  local  detectors 
MPOE  of  S2  with  5  local  detectors 
MPOE  of  S3  with  4  local  detectors  and  2  regional  detectors 
MPOE  of  S4  with  6  local  detectors  and  3  regional  detectors 
MPOE  of  the  centralized  system  with  6  observations 

5.7  Summary  and  Conclusions 

In  this  chapter  we  considered  the  design  and  performance  evaluation  of  four  decen¬ 
tralized  Bayesian  detection  structures.  The  procedure  employed  in  designing  these  sys¬ 
tems  is  based  upon  an  alternate  representation  of  the  minimum  average  cost  in  terms  of  a 
modified  form  of  the  Kolmogorov  variational  distance.  For  the  parallel  fusion  networks 


6.0596  xlO*3 
5.2134  xlO'3 
1.0809x1 O'2 
1.1874  xl0‘2 
1.0998  xlO*3 
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First  solution 


Second  solution 


Threshold  at  LDi 

1.725 

1.77 

Threshold  at  LD2 

1.725 

1.77 

Fusion  rule  at  RDt 

OR 

OR 

Threshold  at  LD3 

0.728 

0.775 

Threshold  at  LD4 

0.728 

0.775 

Fusion  rule  at  RD2 

AND 

AND 

Threshold  at  LD5 

0.725 

1.77 

Threshold  at  LD6 

0.725 

1.77 

Fusion  rule  at  RD3 

AND 

OR 

Permissible  Dn  =  5.23  xlO*3  DI2  =  4.2 28 

Dn  =5.9  xlO'2  DI2  =4.88 

regions  of  Tjj  D12  =  4.228 

D13  =  4.228 

D12  =  4.88  D13  =4.88 

D13  =  4.228 

D14  =  341.42 

D13  =  4.88  D14=  405.05 

Permissible  D2]  =  2.47  xlO*3  D22  =  0.2048 

D21  =2.9  xlO*3  D22  =0.236 

regions  of  ti2  D22  =  0.2048 

D23  =  0.2048 

D22  =0.236  D23  =0.236 

D23  =  0.204S 

!  D24  =  16.97 

D23  =0.236  D24  =19.09 

Permissible  D3l  =  2.47  xlO'3  D32  =  0.2048 

D31  =5.9  xlO'2  D32  =4.88 

region  of  r|3  D32=  0.204 

D33  =  12048 

D32  =4.88  D32  =4.88 

D33  =  0.204 

D^  =16.97 

D33=4.88  D34  =  405.05 

Table  5.4  :  Results  of  Example  5.4  for  the  hierarchical  system  S4 


with  and  without  side  information  at  the  fusion  center,  the  design  of  the  optimum  system 
was  shown  to  reduce  to  the  optimization  of  a  single  function  of  n  variables.  This  optimiza¬ 
tion  is  performed  once  and  does  not  require  an  exhaustive  search.  The  case  of  identical  ob¬ 
servations  at  the  local  detectors  was  considered  in  detail.lt  was  shown  that  the  design 
procedure  requires  the  optimization  of  a  single  function  of  one  variable.  For  the  identical 
observations  case,  the  validity  of  the  identical  decision  rule  (at  local  detectors)  assumption 
was  investigated  by  means  of  an  example.  For  the  example  considered,  optimality  was  not 
lost  when  identical  decision  rules  were  assumed.  For  the  hierarchical  systems  with  and 
without  side  information  at  the  regional  detectors,  the  design  of  the  optimum  system  was 
shown  to  reduce  to  the  optimization  of  a  function  of  3n  variables.  For  the  hierarchical  sys¬ 
tem  with  side  information  in  the  identical  observations  case,  the  local  decision  rules  and 
regional  decision  rules  were  found  to  be  identical.  However,  for  the  hierarchical  system 
without  side  information,  this  was  not  true,  i.e.,  the  decision  rules  associated  with  the  same 
regional  decision  maker  were  identical  but  decision  rules  of  local  detectors  associated 
with  different  regional  decision  makers  were  not  identical.  The  examples  also  illustrate  the 
anticipated  result  that  the  performance  of  hierarchical  systems  is  worse  than  that  of  the 
parallel  systems  due  to  the  extra  loss  of  information  at  the  regional  decision  makers.  In 
this  chapter,  the  validity  of  our  design  methodology  was  demonstrated  by  applying  it  to 
four  decentralized  detection  structures.  It  can  also  be  applied  to  other  decentralized  detec¬ 
tion  topologies  and  structures. 
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CHAPTER  6 


DECENTRALIZED  DETECTION  SYSTEM  DESIGN 
USING  THE  NEYMAN-PEARSON  CRITERION 

6.1  Introduction 

In  Chapters  4  and  5  the  objective  was  to  design  globally  optimum  minimum  average 
cost  decentralized  detection  systems.  A  number  of  system  topologies  were  considered. 
The  design  procedure  required  the  knowledge  of  the  a  priori  probabilities  and  the  costs  in¬ 
curred  by  each  course  of  action.  When  such  information  is  not  available,  the  Neyman- 
Pearson  criterion  may  be  used  for  system  design.  In  this  case,  the  probability  of  false 
alarm  is  restricted  not  to  exceed  a  prespecified  value  a,  and  the  objective  is  to  maximize 
the  probability  of  detection  p.  Srinivasan  [15]  used  the  Neyman-Pearson  criterion  to  ob¬ 
tain  the  local  decision  rules  in  the  parallel  fusion  network  assuming  that  the  fusion  center 
is  a  combinational  logic  circuit.  Hoballah  and  Varshney  [16]  treated  the  problem  in  two  re¬ 
spects.  First,  when  the  fusion  rule  is  known  and  the  objective  is  to  find  the  local  decision 
rules.  Second,  when  the  decision  rules  at  the  detectors  are  given,and  the  objective  is  to  find 
the  optimum  fusion  rule.  Thomopoulos,  Viswanathan,  and  Bougoulias  [17]  employed  the 
Neyman-Pearson  criterion  for  system  design  where  both  the  decisions  made  by  the  indi¬ 
vidual  sensors  and  the  global  decision  made  by  the  fusion  center  are  based  on  the  Ney¬ 
man-Pearson  test.  Tsitsiklis  [18]  addressed  the  question  of  concavity  of  the  receiver 
operating  characteristic  of  the  system.  He  found  that  for  a  given  strategy,  the  receiver  op¬ 
erating  characteristic  is  not  necessarily  a  concave  function  (  a  numerical  example  is  pro¬ 
vided  in  [19] ).  However,  concavity  can  be  achieved  by  randomizing  with  respect  to  the 
possible  strategies.  A  similar  result  was  obtained  by  Willet  and  Warren  [20].  Viswanathan, 
Thomopoulos,  and  Tumuluri  [21]  applied  the  Neyman-Pearson  criterion  to  the  design  of 
the  serial  decentralized  configuration.  They  found  that  for  the  case  of  two  sensors,  the  op¬ 
timal  serial  network  has  a  better  performance  than  the  parallel  scheme  (  better  here  refers 
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to  higher  probability  of  detection  for  the  same  false  alarm  probability).  While  this  interest¬ 
ing  result  is  true  for  the  case  of  two  sensors,  the  numerical  examples  provided  in  [21] 
show  that  tins  result  is  not  true,  in  general,  for  systems  with  more  than  two  "ensors. 

As  mentioned  earlier,  the  final  decision  in  the  parallel  fusion  system  of  Fig.  1 . 1  is 
based  on  the  decision  vector  U  =  [zj  Z2  ...  zn].  The  observation  space  corresponding  to  this 
decision  vector  is  ,  of  course,  discrete.  When  the  observation  space  in  a  hypothesis  testing 
problem  is  discrete,  questions  related  to  the  randomization  of  the  decision  rule  arise.  In 
Chapter  2  we  addressed  the  question  of  randomization  when  the  decision  maker  was  as¬ 
sumed  to  be  a  minimum  average  cost  receiver.  In  that  chapter  we  found  that  randomization 

of  the  decision  rule  is  not  necessary.  This  idea  was  implemented  in  Chapters  4  and  5  to  de 

« 

sign  globally  optimum  minimum  average  cost  decentralized  detection  systems  without  the 
need  to  randomize  the  decision  rule.  This  is  not  necessarily  the  case  wnen  the  design  crin- 
ri„n  is  the  .deyman -Pearson  criterion.  In  this  chapter,  the  objective  is  to  investigate  wheth¬ 
er  randomization  is  necessary  at  the  fusion  center  in  the  design  of  the  Neyman -Pearson 
parallel  fusion  system  .  Howevei,  the  objective  is  not  to  provide  a  c  implete  design  ap¬ 
proach  to  the  Nevman-Pearson  decentralized  detection  system.  In  Section  6.2  we  formu¬ 
late  the  problem.  In  Section  6.3  we  consider  the  problem  of  randomization  in  detail  and 
show  that  randomization  is  not  necessary.  In  Section  6.4  we  discuss  the  results  obtained  in 
this  chapter. 

6.2  Problem  Formulation 

Consider  the  parallel  fusion  system  shown  in  Figure  1.1  consisting  of  n  local  detec¬ 
tors  and  a  fusion  center.  The  system  receives  n  ojservations  Xj,  Xj Xn  where  X;  de¬ 
notes  the  observation  received  by  the  local  detector  LDj.  We  assume  that  these 
observations  are  independent  and  identically  distributed  random  variables  with  condition¬ 
al  probability  density  functions  Pj(x,),  j  =  0,  1,  i  =  1, ...,  n.  We  make  the  srme  assumption 
that  we  made  in  Chapter  4  in  that  the  local  detectors  are  identical,  each  :  ne  characterized 
by  a  probability  of  false  alarm  Pp  and  a  probability  of  detection  PD.  The  operating  point 
(Pp,  Pp  )  can  be  anywhere  on  the  receiver  operating  characteristic  of  the  local  detectors. 
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Let  K  be  the  random  variable  representing  the  number  of  sensors  that  decide  in  favor  of 
Hj.  The  conditional  probability  density  functions  of  K  under  hypotheses  Hq  and  Hj  are 
given  by  (4.34)  and  (4.35)  respectively.  Let  $(k)  be  the  probability  of  accepting  Hj  after 
observing  K  =  k.  Then,  in  general,  0(k)  is  given  by  (see  Appendix  A  equation  (A.  13)) 

{1  when  k  >  k* 

y  when  k  =  k*  (6.1) 

0  otherwise 

where  y  is  the  randomization  factor  and  k*  is  the  threshold.  The  randomization  factor  in 
(6.1)  is  introduced  so  as  to  achieve  the  constraint  on  the  global  false  alarm  probability  a. 
The  probability  of  false  alarm  a  is  the  expected  value  of  4>(k)  when  Hq  is  true,  while  the 
probability  of  detection  (3  is  the  expected  value  of  <p(k)  when  Hj  is  true.These  quantities 
are  therefore  expressed  as 


a  =  E0  { <t>  (k)  } 


(6.2) 


P  =  £!(«>(*)} 


(6.3) 


Using  (4.34)  and  (4.35),  we  obtain  the  following  expressions  for  a  and  (3 


a  = 


n  ( 

\ Kk)rF^'rF> 


P%  {\-PF)n~k*  +  £  I  PkF{\-PF)n-k 


(6.4) 


k  =  e  +i 


k  =  k*  +1 


(  ^ 

n 

\kj 


PdU-Pd) 


n  -  k 


(6.5) 


When  the  local  decision  rules  are  known,  the  randomization  factor  and  the  threshold  k* 
can  be  easily  determined  using  (6.4)  to  satisfy  the  desired  probability  of  false  alarm.  The 
values  of  k*  and  y  that  satisfy  (6.4)  are  used  to  find  the  probability  of  detection  in  (6.5). 


129 


The  resulting  (5  is  the  maximum  that  can  be  achieved.  When  the  local  decision  rules  are 
not  known,  and  the  objective  is  to  design  the  system  that  maximizes  (3  for  the  specified  a, 
the  problem  becomes  involved.  In  this  case,  only  the  constraint  on  the  system  probability 
of  false  alarm  is  given  while  three  parameters  have  to  be  determined,  namely,  the  thresh¬ 
old  k*,  the  randomization  factory,  and  the  local  threshold.  For  this  problem,  we  will  in¬ 
vestigate  the  role  of  the  randomization  factor  y  in  the  overall  design  process. 

6.3  Randomization  in  Decentralized  Neyman-Pearson  Detection  Systems 

In  this  section  we  show  that  when  the  global  false  alarm  probability  is  specified  at 
some  level  a,  then  the  global  probability  of  detection  (5  in  (6.5)  is  maximized  when  the  pa¬ 
rameter  y  is  either  0  or  1.  We  begin  by  assuming  that  at  some  point  (say  poir ♦  PI  in  Fig. 
6.1)  on  the  receiver  operating  characteristic  (ROC)  of  an  individual  detector,  the  specified 
a  is  achieved  when  the  randomization  factor  in  (6.4)  is  1.  This  means  that  the  fusion  rule 
at  the  fusion  center  is  a  k*  out  of  n  rule.  In  general,  a  k  out  of  n  fusion  rule  is  defined  as 
follows  :  The  global  decision  maker  decides  in  favor  of  hypothesis  Hj  when  the  number  of 
local  detectors  that  decide  in  favor  of  hypothesis  is  greater  than  or  equal  to  k.  It  is 
shown  in  Appendix  A  that  when  the  fusion  center  is  a  minimum  average  cost  receiver, 
then  under  the  identical  detector  assumption  the  optimum  fusion  rule  is  a  k  out  of  n  rule. 
Let  the  local  detection  and  false  alarm  probabilities  at  the  point  PI  be  denoted  by  Pqj  and 
Ppi,  and  let  the  local  threshold  that  achieves  these  probabilities  be  denoted  by  X\  (see 
equation  (4.5) ).  Now,  assume  that  a  is  also  achieved  at  two  neighboring  points  PO  and  P2 
as  shown  in  Fig.  6.1  with  corresponding  global  detection  probabilities  Po  and  02  respec¬ 
tively.  Let  the  local  thresholds  at  the  points  PO  and  P2  be  denoted  by  Tq  and  X2-  Now  we 
prove  the  following  theorem. 

Theorem  6.1 

In  designing  a  globally  optimum  Neyman-Pearson  decentralized  parallel  fusion  sys¬ 
tem,  the  global  probability  of  detection  is  maximized  when  the  randomization  factor  at  the 
fusion  center  with  respect  to  the  local  decisions  is  either  0  or  1,  i.e.,  system  without  ran¬ 
domization  performs  better  than  the  ones  employing  randomization. 
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Proof 


To  prove  the  theorem  we  need  to  show  that  Pi>Po  Pi>P2  •  The  proof  will  be  car¬ 
ried  out  in  two  parts.  In  the  first  part  we  show  that  Pi>Po  ^d  in  the  second  we  show  that 

Pl>p2- 

Part  1 :  Pi>Po 

Since  at  point  PI,  a  is  satisfied  by  a  k  out  of  n  fusion  rule  with  k  =  k*,  the  random¬ 
ization  factor  in  (6.1)  is  1. Therefore  (6.4)  and  (6.5)  reduce  to 

<*=  i  mpM'-'Vi)"-'  (66) 

>  =  ** 


Pi=  I  uKiO-^di)"-' 

/  =  ** 


(6.7) 


Now  we  show  that  for  two  positive  numbers  0  <  yj  <  1  and  0  <  y2  <  1  such  that  yl  < 
y2,  the  following  inequality  is  true 


I  I  my2(i-y2)n"> 

j=?  j= 


(6.8) 


We  prove  the  inequality  in  (6.8)  by  showing  that  the  slope  of  the  following  function  of  y  is 
positive 


g (y)  =  I  (nijy'(i-y)n-J 

j  =  ? 


(6.9) 


Let  us  define  the  function  gj(y)  as 


gj (y)  =  yid-y)n  3 


(6.10) 
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The  derivative  of  gj(y)  is 


fyj(y)  =  jy>~1  d-y)n-j -  (n-j)y> d-y)^-1  (6.11) 


Using  (6.10)  and  (6.11),  the  derivative  of  the  function  g(y)  defined  in  (6.9)  becomes 


»  (  ^ 

'  (rv 


j-g(y)  =  S(y)  =  X  (7 J {/y-1  <1  (»-y>y(l-y)"_>-,> 

j  =  £ 


(6.12) 


It  can  be  shown  that  the  summation  S(y)  in  (6.12)  simplifies  to 


S(y)  = 


n\ 

(t  ~  1) !  (n-  k*)\ 


/_1(i -y)n~e 


(6.13) 


The  summation  in  (6.13)  is  positive  for  all  the  values  of  k*  when  the  values  of  y  are  in  the 
range  0  <  y  <  1.  This  result  shows  that  the  slope  of  the  function  g(y)  is  positive  indicating 
that  g(y)  is  an  increasing  function  of  y  in  the  range  0  <  y  <  1.  Therefore,  for  yj  <  y2, 
g(yi)  <  g(y2)  as  claimed  in  (6.8). 


From  Fig.  6.1  we  see  that  Ppg  <  PF1.  Therefore,  using  (6.8)  we  conclude  that 


(6.14) 


Using  (6.6),  (6.14)  can  be  written  as 


£  -PF0)"-'< a  (6.15) 

j=k* 


In  order  to  maintain  a  false  alarm  probability  of  a  at  point  P0,  a  randomization  term  at  k  = 
k*-l  has  to  be  added  to  the  left  hand  side  of  (6.15)  so  that  equality  is  achieved.  Therefore, 
the  following  decision  rule  is  employed  at  the  point  P0 
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{1  when  k>k*-l 
Yo  when  k  =  k*-l 
0  otherwise 


(6.16) 


Using  (6.16)  we  evaluate  the  global  false  alarm  and  the  global  detection  probabilities  at 
the  point  PO  as 

«  =  *»(*»-  1)Pfo'1<l-'Vo>"-':’+1+  I  ("^Fod-fFo)"-' 

j  ~  £ 

(6.17) 

(  \  ,  n  f  \ 

Po  =  %(**_,  /do-1  S  Ij  J^oo  <  1  -  /’do)  "  “j 

j  =  k* 

(6.18) 

From  (6.17),  we  find  that  y0  is  given  by 

(6.19) 

Substituting  (6.19)  into  (6.18)  we  get 

p»=  t  (xKod-'w"-' 

j=km 


«- 1  •noo-'w"- 


n  pk*  - 1 


+1 


(6.20) 


Substituting  the  value  of  a  given  in  (6.6)  into  (6.20)  we  get 


Po  ~  X  [j  /do  ( 1  pdq)  n  J  + 

j  =  k* 


Pn*\k*  ~l  rl-Pnn\n~k*  +1 


<  i  \j  J  V*Fl  ( 1  -  ^Fl)  j  (  1  -  *Vo)  n~']  > 

j  =  £  V  '  VFO  J  \  1  “  ‘  FO ) 


(6.21) 
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Our  objective  now  is  to  express  the  right  hand  side  of  (6.21)  in  terms  of  (5j.  For  this  pur¬ 
pose  we  recall  the  following  approximation  based  on  Taylor  series  expansions  of  a  func¬ 
tion  g(y) 

g  O’  +  Ay)  =  g  ( y )  +  g'  (y)  Ay  (6.22) 


g(y-Ay)  =g  (y)  -g'(y)Ay 


(6.23) 


In  the  rest  of  the  chapter  the  approximation  sign  in  (6.22)  and  (6.23)  will  be  replaced  by 
equality  sign.  Let  us  consider  the  function  g(y)  given  by  (6.9).  The  derivative  of  this  func¬ 
tion  is  given  by  (6.12).  Using  (6.22)  and  (6.23)  we  get  the  following  expansions  of  the 
function  g(y) 


s(y  +  Ay)  =  £  [jj/d-y)"  j  + 

>  =  ** 


{  X  (JJ{yy"1a-F)n^-(«-y)y(i->')n';-1}}Ay  (6.24) 

j  =  k* 


g(y-Ay)  =  £  (jj/O-y)”  J  - 

j  =  k* 

{  X  (/,  1  O  -y)n~j- («-7')y(l  -y)n~J~1}  }  Ay  (6.25) 

j  =  * 

Since  Ppj  =  Ppg  +  APp  (refer  to  Fig.  6.1),  then  by  making  use  of  (6.24)  we  obtain  the  fol- 
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lowing  expansion 


(jr'm  '(i-PFO)"-J-('‘-j)riFo(1-pFo)"~,~''>  *Pf 


(6.26) 


Substituting  (6.26)  into  (6.21)  we  get 


n~j  + 


~  pdo' 
V  1  ~  PF0  , 


k*  +  ] 


j  =  k* 


(6.27) 


Since  Pix)=pDr^pD’ tflcn  we  can  use  (6.25)  to  obtain  the  following  expansion 


i  (/Wifi1  <1  (n-j)P'Di(l-P[H)n-’-')APD 

j  =  k* 

(6.28) 

It  can  be  easily  recognized  that  the  first  term  on  the  right  hand  side  of  (6.28)  is  the  global 
probability  of  detection  at  the  point  PI  as  given  by  (6.7).  Substituting  (6.28)  into  (6.27) 
and  making  use  of  (6.7)  we  get 

Po  =  P,-S<P01)A/>0  +  S<Pfo)A/v(^)  (1=§) 

(6.29) 
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where  S(.)  is  given  by  (6.12).  Making  use  of  (6.13),  equation  (6.29)  simplifies  to 


Po 


n\ 

(k*  -l)\(n-k*)\ 


x 


( 1  - pD l ) ’■ - e  AZ-jr *PF 1  ( 1  - PF0) "■ - * 


k*  +  1  - 


(6.30) 


In  the  limit  when  APp  and  APp  approach  zero,  (6.30)  becomes 


Po 


n\ 

{k*  - \)\{n-k*)\ 


'-Ppi 
1  ~PF  j 


(6.31) 


But  the  following  inequality  is  true  for  any  operating  point  (Pp,Pj>)  on  the  receiver  operat¬ 
ing  characteristic  (see  Appendix  D  for  a  proof) 


APD(l-PF)>bPF(\-PD ) 


(6.32) 


Using  this  inequality,  we  see  that  the  term  between  brackets  on  the  right  hand  side  of 
(6.31)  is  positive,  and  so  we  establish  the  following  result 

P0<Pi  (6-33) 


This  concludes  the  first  part  of  the  proof.  Now  we  consider  the  second  part. 


Part  2 :  Pi>p2 

From  Fig.  6.1,  we  note  that  the  false  alarm  probability  Pp2  at  point  P2  is  greater  than 
the  false  alarm  probability  Pp]  at  point  PI.  Therefore,  using  (6.8)  we  conclude  that 


I  I  iy. 

j  =  k*  i  =  k * 


H-iU-'Vi)"-' 


F\> 


(6.34) 
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Using  (6.6)  we  can  rewrite  (6.39)  as 


j  =  k* 


(6.35) 


In  order  to  maintain  a  false  alarm  probability  of  a  at  point  P2,  the  outcome  at  k  =  k*  on  the 
left  hand  side  of  (6.35)  is  randomized  so  as  to  achieve  equality  in  (6.35).  Therefore,  the 
following  decision  rule  is  employed  at  the  point  P2 


1  when  k  >  k* 
0(k)  =  {  Y2  when  k  =  k* 
0  otherwise 


{ 


(6.36) 


Using  (6.36)  we  evaluate  the  global  false  alarm  and  the  global  detection  probabilities  at 
the  point  P2  as 


a  = 


n  (  ^ 

n-k*  ,  In 


I  lyj'ko-'w- 


-j 


j  =  k*  +1 


(6.37) 


p2  =  Y2^J^2(l-^02)”-i*+  I  \j)pjD2^-PDl)n~i 

j  =  £  +l 


(6.38) 


From  (6.37),  we  find  that  y2  is  given  by 


%  G" 

:  _  tr 


PjF2^-PF2)n-J 


y  =  - i  j  £ X  1 

72  U  nk* 


\rnU-PFiV-k' 


(6.39) 


Substituting  (6.39)  into  (6.38)  we  get 


P2  =  S  i/H2(l-^D2)"'7 

j  =  k*  +1 


1  /Hid -rn)"-i) 


■pD2/  r\-p,  '"-4‘ 


j  =  k*  +1 


F  2 


D  2 


V  1  “  PF2 


(6.40) 
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Substituting  the  value  of  a  given  in  (6.6)  into  (6.40)  we  get 


j  =  k*  +1 


02=  I  )H2(l-/>C2>"“y 


W  n-pD2^'e 


PF2  )  V  1  ~  PF2 


i  X  [})pjFlV-pjF0n~J-  X  I'M piF2^-pjF2)n~J}  (6.41) 

j  -  ^  +  i 


As  we  did  in  the  first  part,  here  our  objective  is  to  express  in  terms  of  pj.  Since  Ppj 
Pp2  -  APp ,  then  by  making  use  of  (6.25)  we  obtain  the  following  expansion 


j  =  k*  j  =  k* 


X  17  J  VpJF2 1  (1  -  PF2)  n’s-(n-j)  pint  1  -  PF2) "  ' 1 )  A  PF 

j  =  k*  V 


Substititing  (6.42)  into  (6.41)  we  get 


(6.42) 


j  =  k  +1 


P2=  X  [jpjD2('-pD2)n-j+  [y1]  y 


V2  J  l 1  PF2  J 


<lZjpkF2(1-pF2)n~k*  ~S(PF2)ApF} 


(6.43) 


where  S(.)  is  given  by  (6.12).  In  obtaining  (6.43)  we  have  made  use  of  the  fact  that 


j  =  k*  +  1 


I  \jW(l-y)’-J-  £  7V(i-y)->-  a 


(6.44) 


Equation  (6.43)  can  also  be  written  as 
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P2  =  I  \!])pjD2^-  PD2)  "  -j  +  U*  jPD2  0  -  PD2) 

j=k*  +  1 


n-e 


~S(PF2) 


(P 


D2 


\P  F2 


f  f'-PD2 

J  l'-PF2. 


.n-k* 


A  Pr 


(6.45) 


Using  the  expression  for  S(.)  given  by  (6.13),  equation  (6.45)  reduces  to 


P2  =  I  [ni)piD2^-PD2)n 

j  =  k* 


-J 


pi '*  —  1  /  -j  _  p  \n~  £*  (  1  ^2 

(k*  -  l)lTn-k*)\  F1  (  Fl)  lir~ 


.*  ( P ni)  /T  —  P 


.n-k* 


F2J 


D  2 


1  -P 


A  PL 


F  2 


(6.46) 


Now  expand  the  first  term  on  the  right  hand  side  of  (6.46)  about  the  point  PD1 

nk* 


p2  =  X 


n! 


j  =  k 


(k*  -  1)!  (n-k  )!  P 


D2(l-PD2)n-k*  APf 
F2 


+  X  \j)VpJDi(l-pDi)n-j-(n-j)PiDl(l-PD1)n-j-l]APD 

j  =  k* 


(6.47) 


The  first  term  on  the  right  hand  side  of  (6.47)  is  Using  (6.13),  equation  (6.47)  becomes 

Pk* 

p2  =  p,  - 


n\ 


(F  -l)\(n-k*)\PF2 


D2(l-PD2)n-k*  &PF 


n! 


(6.48) 


As  point  P2  becomes  closer  and  closer  to  point  PI,  equation  (6.48)  can  be  expressed  as 
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(6.49) 

But  for  any  point  (Pp,  Pp)  on  the  receiver  operating  characteristic  of  an  individual  sensor, 
the  following  inequality  is  true  (see  Fig.  6.2  and  Appendix  D) 

P pA P p>  P pAP p  (6.50) 

Making  use  of  this  inequality,  we  see  that  the  second  term  on  the  right  hand  side  of  (6.49) 
is  positive  which  indicates  that 

P2<3i  (6-51) 

This  completes  the  proof  that  randomization  of  the  observations  at  the  fusion  center  is  not 
necessary  when  the  objective  is  to  design  a  globally  optimum  Neyman-Pearson  decentral¬ 
ized  detection  system. 

6.4  Summary 

In  this  chapter  we  addressed  the  problem  of  randomization  with  respect  to  the  local 
decisions  at  the  fusion  center  when  the  objective  is  to  design  an  optimum  Neyman-Pear¬ 
son  decentralized  detection  system.  We  found  that  for  a  prespecified  value  of  the  global 
probability  of  false  alarm  the  global  probability  of  detection  is  maximized  when  there  is 
no  randomization,  i.e.,  the  randomization  factor  is  either  0  or  1.  This  means  that  a  k  out  of 
n  fusion  rule  is  the  fusion  rule  that  maximizes  the  global  probability  of  detection  ,i.e.,  the 
global  probability  of  detection  has  (k+1)  local  maximum  points  for  the  specified  false 
alarm  probability.  One  remaining  aspect  of  the  problem  is  to  determine  on  theoretical  ba¬ 
sis  the  particular  fusion  rule  with  the  largest  probability  of  detection. 


1 


PF 

Fig.  6.2.  Due  to  the  concavity  of  the  receiver  operating  characteristic,  Q\  <  02 
where  0i=  dPD  /  dPp  is  the  slope  and  02  =  PD  /  PF  • 


CHAPTER  7 


SUMMARY  AND  SUGGESTIONS  FOR  FUTURE 

WORK 


7.1  Summary 

In  this  dissertation,  we  considered  the  design  and  performance  evaluation  of  distrib¬ 
uted  detection  networks  mainly  from  a  Bayesian  viewpoint.  In  a  distributed  detection  sys¬ 
tem,  a  group  of  local  detectors  process  the  observations  they  receive  about  the  status  of  a 
certain  phenomenon,  and  transmit  their  decisions  to  a  fusion  center  where  the  final  deci¬ 
sion  is  made.  Due  to  constraints  on  the  channel  capacities  linking  the  local  detectors  to  the 
global  decision  maker,  the  local  detectors  compress  their  local  observations  to  a  hard  or  a 
soft  decision  and  transmit  this  decision  to  the  fusion  center  where  the  global  decision  is 
made.  The  design  issues  related  to  the  distributed  detection  systems  involve  specifying  the 
local  decision  rules  and  the  global  decision  rule.  We  presented  a  computationally  efficient 
approach  to  the  design  of  decentralized  Bayesian  detection  systems.  This  procedure  is 
based  upon  an  alternate  representation  of  the  minimum  average  cost  in  terms  of  a  modified 
form  of  the  Kolmogorov  variational  distance.  We  demonstrated  the  utility  of  our  approach 
by  applying  it  to  the  design  and  performance  evaluation  of  four  decentralized  detection 
structures.  In  all  these  structures,  the  design  of  the  optimum  systems  reduced  to  the  opti¬ 
mization  of  a  single  function  of  a  certain  number  of  variables.  This  optimization  is  per¬ 
formed  once  and  does  not  need  an  exhaustive  search.  Two  methods  that  deal  with  the 
design  of  binary  decentralized  Bayesian  detection  systems  are  reported  in  the  literature.  In 
the  first  method,  the  fusion  center  is  fixed  and  a  set  of  n  coupled  nonlinear  equations  are 
solved  to  deteimine  the  n  local  thresholds.  This  has  to  be  repeated  for  all  the  permissible 
fusion  rules.  The  solution  with  the  smallest  overall  cost  is  finally  selected  as  the  optimum 
system.  The  exponential  growth  of  the  number  of  fusion  rules  to  be  searched  makes  the 
use  of  this  method  impractical.  The  other  method  is  the  person-by-person  optimization 
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procedure  which  need  not  yield  the  globally  optimum  solution.  This  procedure  requires  a 
simultaneous  solution  of  (2n+n)  coupled  nonlinear  equations  for  the  binary  hypothesis 
testing  problem.  Our  design  procedure  requires  the  minimization  of  a  single  function  of  n 
variables  where  n  is  the  number  of  local  detectors.  This  is  computationally  simpler  and  ef¬ 
ficient  optimization  algorithms  can  be  employed  to  design  relatively  large  decentralized 
detection  structures. 

The  performance  degradation  of  distributed  detection  systems  relative  to  the  central¬ 
ized  system  was  determined.  It  was  also  shown  that  the  performance  of  the  decentralized 
system  approaches  the  performance  of  the  centralized  system  very  quickly  as  a  function  of 
the  number  of  quantization  levels.  In  addition,  the  performance  of  the  optimum  systems 
was  compared  to  the  performance  of  suboptimum  systems  designed  based  on  criteria  other 
than  the  global  optimum  Beyesian  cost. 

We  derived  upper  and  lower  bounds  on  the  minimum  probability  of  error  and  the 
minimum  average  cost  for  optimum  receivers.  Some  of  the  bounds  on  the  minimum  aver¬ 
age  cost  are  generalizations  of  the  known  bounds  on  the  minimum  probability  of  error  in 
terms  of  the  Bhattacharyya  bound  and  the  Chemofif  bound.  Furthermore,  we  derived  a  new 
upper  bound  on  the  probability  of  error  which  is  tighter  than  the  previously  available 
bounds.  This  bound  was  applied  to  design  a  nearly  optimum  decentralized  detection  sys¬ 
tem.  Tight  Lower  bounds  on  the  probability  of  error  and  the  minimum  average  cost  were 
also  derived  in  terms  of  the  new  upper  bound. 

We  addressed  the  problem  of  randomization  with  respect  to  the  local  decisions  at  the 
fusion  center  when  the  objective  is  to  design  an  optimum  Neyman-Pearson  decentralized 
detection  system.  We  found  that  for  a  prespecified  value  of  the  global  probability  of  false 
alarm,  the  global  probability  of  detection  is  maximized  when  there  is  no  randomization, 
i.e.,  the  randomization  factor  is  either  0  or  1. 

7.2  Suggestions  for  Future  Work 

In  this  section,  we  discuss  some  of  the  problems  related  to  the  topics  treated  in  this 
dissertation  and  may  be  pursued  in  the  future. 
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1)  In  Chapter  4,  we  employed  the  new  upper  bound  derived  in  Chapter  3  to  design  a  nearly 
optimum  decentralized  detection  system.  A  generalization  of  the  problem  is  to  employ  the 
new  upper  bound  for  the  design  of  an  M-level  quantizer  for  hypothesis  testing. 

2)  In  decentralized  detection  systems,  usually  there  are  capacity  constraints  on  the  chan¬ 
nels  linking  the  local  detectors  to  the  destination.  An  interesting  problem  would  be  to  ex¬ 
amine  from  an  information  theoretic  viewpoint,  the  design  of  decentralized  detection 
systems  when  capacity  constraints  are  placed  on  the  channels.  For  example,  the  output  en¬ 
tropy  may  be  restricted  to  be  less  than  some  prespecified  value.  One  then  needs  to  find  the 
fusion  rule  and  the  local  decision  rules  under  this  constraint. 

3)  In  Chapter  6,  we  investigated  the  role  of  the  randomization  factor  in  the  decentralized 
Neyman-Pearson  detection  problem.  We  considered  the  case  when  the  local  detectors 
were  identical  and  they  employed  identical  decision  rules.  We  found  that  randomization 
was  not  necessary  in  the  design  of  the  optimum  system.  Two  problems  related  to  this  re¬ 
sult  arise.  The  first  is  to  determine,  based  on  a  theoretical  basis,  the  fusion  rule  with  the 
highest  probability  of  detection  for  the  specified  probability  of  false  alarm.  The  second  is 
to  show  that  this  result  is  true  for  nonidentical  local  detectors. 


145 


REFERENCES 


[1]  H.L. Van  Trees,  Detection,  Estimation  and  Modulation  Theory,  Vol.  1,  Wiley,  1968. 

[2]  H.V.Poor,  An  Introduction  to  Signal  Detection  and  Estimation ,  Springer- Verlag,  N.Y. 
1988. 

[3]  R.E.Ziemer  and  W.H.Tranter,  Principles  of  Communications,  Systems,  Modulation, 
and  Noise,  Houghton  Mifflin,  1985. 

[4]  J.N.Tsitsiklis  and  M.Athans,  "On  the  Complexity  of  Decentralized  Decision  Making 
and  Detection  Problems,"  IEEE  Transactions  on  Automatic  Control,  Vol.  30,  pp.  440-446, 
May  1985. 

[5]  R.R.Tenney  and  N.R.Sandell,  “  Structures  of  Distributed  Decisionmaking,”  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics,  Vol.  11,  pp.  517-526,  Aug.  1981. 

[6]  R.R.Tenney  and  N.R.Sandell,  “  Strategies  for  Distributed  Decisionmaking,”  IEEE 
Transactions  on  Systems,  Man,  and  Cybernetics ,  Vol.  11,  pp.  527-538,  Aug.  1981. 

[7]  R.R.Tenney  and  N.R.Sandell,  "  Detection  with  Distributed  Sensors,"  IEEE  Transac¬ 
tions  on  Aerospace  and  Electronic  Systems,  Vol.  17,  pp.  501-510,  July  1981. 

[8]  F.A.Sadjadi,  “  Hypothesis  Testing  in  a  Distributed  Environment,”  IEEE  Transactions 
on  Aerospace  and  Electronic  Systems,  Vol.  22,  pp.  134-137,  March  1987. 

[9]  Z.Chair  and  P.K.  Varshney,  "Optimal  Data  Fusion  in  Multiple  Detection  Systems," 
IEEE  Transactions  on  Aerospace  and  Electronic  Systems  Vol.  22,  pp.  98-101,  Jan.  1986. 

[10]  I.Hoballah  and  P.K. Varshney,  "Decentralized  Bayesian  Signal  Detection,"  IEEE 
Transactions  on  Information  Theory,  Vol.  35,  pp.  995-1000,  Sep.  1989. 

[11]  Z.B.Tang,K.R.Pattipati,and  D.L.Kleinman,  "An  Algorithm  for  Determining  the  Deci¬ 
sion  Thresholds  in  a  Distributed  Detection  Problem,"  Preprint,  University  of  Connecticut. 

[12]  A.R.Reibman  and  L.W.Nolte,  "Optimal  Detection  and  Performance  of  Distributed 
Sensor  Systems,”  IEEE  Transactions  on  Aerospace  and  Electronic  Systems,  Vol.  23,  pp. 


146 


24-30,  Jan.  1987. 


[13]  S.C.A.Thomopoulos,R.Viswanathan,and  D.K.Bougoulias,  "Optimal  Distributed  De¬ 
cision  Fusion,"  IEEE  Transactions  on  Aerospace  and  Electronic  Systems,  Vol.  25,  pp.  761- 
765,  Sep.  1989. 

[14]  Z.Chair  and  P.K.Varshney,  "Distributed  Bayesian  Hypothesis  Testing  with  Distribut¬ 
ed  Data  Fusion,"  IEEE  Transactions  on  Systems,  Man,  and  Cybernetics,  Vol.  18,  pp.  695- 
699,  Sep.  /Oct.  1988. 

[15]  R.Srinivasan,  "Distributed  Radar  Detection  Theory,"  IEE  Proceedings,  Vol.  133-F, 
pp.  55-60,  Feb.  1986. 

[16]  I.Y.Hoballah,  “  On  the  Design  and  Optimization  of  Distributed  Signal  Detection  and 
Parameter  Estimation  Systems,”  PhD  Dissertation,  Syracuse  University,  1986. 

[17]  S.C.A.Thomopoulos,R.Viswanathan,and  D.K.Bougoulias,  “  Optimal  Decision  Fu¬ 
sion  in  Multiple  Sensor  Systems,”  IEEE  Transactions  on  Aerospace  and  Electronic  Sys¬ 
tems,  Vol.  23,  pp.  644-654,  Sep.  1987. 

[18]  J.N.Tsitsiklis,  “  Extremal  Properties  of  Likelihood  Ratio  Quantizers,”  Preprint,  Labo¬ 
ratory  for  Information  and  Decision  Systems,  Massachusetts  Institute  of  Technology,  Nov. 
1989. 

[19]  A.Reibman,  “  Performance  and  Fault-Tolerance  of  Distributed  Detection  Networks,” 
PhD  Dissertation,  Department  of  Electrical  Engineering,  Duke  University,  Durham,  North 
Carolina,  1987. 

[20]  P.Willet  and  D.Warren,  “  The  Suboptimality  of  Randomized  Tests  in  Distributed  and 
Quantized  Detection  Systems,”  Preprint,  Nov.  1989. 

[21]  R.Viswanathan,S.C.A.Thomopoulos,and  R.Tumuluri,  "Optimal  Serial  Distributed 
Decision  Fusion,"  IEEE  Transactions  on  Aerospace  and  Electronic  Systems,  Vol.  24,  pp. 
366-376,  July  1988. 

[22]  L.K.Ekchian  and  R.R.Tenney,  "Detection  Networks,"  Proceedings  of  the  21st  IEEE 
Conference  on  Decision  and  Control,  Orlando,  Fla.,  pp.  686-691,  Dec.  1982. 


147 


[23]  A.R.Reibman  and  L.W.Nolte,  "Design  and  Performance  Comparison  of  Distributed 
Detection  Networks,"  IEEE  Transactions  on  Aerospace  and  Electronic  Systems,  Vol.  23, 
pp.  789-797,  Nov.  1987. 

[24]  J.N.Tsitsiklis,  "Decentralized  Detection,"  Technical  Report  LIDS-P-1913,  M.I.T., 
Sep.  1989. 

[25]  P.K.Varshney,  “  Decentralized  Bayesian  Detection  Theory,”  Invited  Chapter  to  appear 
in  the  Book  Large  -  Scale  Stochastic  Systems:  Detection,  Estimation,  Stability  and  Con¬ 
trol,  S.G.Tzafestas  and  K.Watanabe,  Editors,  Marcel  Dekker  Inc. 

[26]  J.D.Papastavrou  and  M.Athans,  "On  Optimal  Distributed  Decision  Architectures  in  a 
Hypothesis  Testing  Environment,"  Technical  Report  LIDS-P-1928,  M.I.T.,  Jan.  1990. 

[27]  P.Willet  and  D.Warren,  "Randomization  and  Scheduling  in  Decentralized  Detection," 
Proceedings  of  the  27  *  Annual  Conference  on  Communication,  Control,  and  Computing, 
Sep.  1989. 

[28]  E.Drakopoulos  and  C.C.Lee,  “Optimum  Multisensor  Fusion  of  Correlated  Local  De¬ 
cisions,”  Preprint,  Department  of  Electrical  Engineering  and  Computer  Science,  North¬ 
western  University. 

[29]  G.Polychronopoulos  and  J.Tsitsiklis,  “  Explicit  Solutions  for  Some  Simple  Decentral¬ 
ized  Detection  Problems,”  IEEE  Transactions  on  Aerospace  and  Electronic  Systems,  Vol. 
26,  pp.  282-292,  March  1990. 

[30]  S.M.Ali  and  S.D.Silvey,  “  A  General  Class  of  Coefficients  of  Divergence  of  One  Dis¬ 
tribution  From  Another,”  J Royal  Stat.  Soc.,  Series  B,  Vol.  28,  pp.  131-143,  1966. 

[31]  T.Kailath,  ’The  Divergence  and  Bhattacharyya  Distance  Measures  in  Signal  Selec¬ 
tion,”  IEEE  Transactions  on  Communication  Technology,  Vol.  15,  pp.  52-60,  Feb.  1967. 

[32]  H.V.Poor  and  J.B.Thomas,  ’’Applications  of  Ali-Silvey  Distance  Measures  in  the  De¬ 
sign  of  Generalized  Quantizers  for  Binary  Decision  Systems,”  IEEE  Transactions  on 
Communications,  Vol.  25,  pp.  893-900,  Sep.  1977. 

[33]  H.V.Poor,  “  Fine  Quantization  in  Signal  Detection  and  Estimation,”  IEEE  Transac- 


148 


tions  on  Information  Theory,  Vol.  34,  pp.  960-972,  Sep.  1988. 

[34]  G.R.Benitz  and  J.A.Bucklew,  “  Asymptotically  Optimal  Quantizers  for  Detection  of 
I.I.D.  Data,”  IEEE  Transactions  on  Information  Theory,  Vo.  35,  pp.  316-325,  March  1989. 

[35]  M.Longo,T.Lookabaugh,and  R.M.Gray,  ’’Quantization  for  Decentralized  Hypothesis 
Testing  Under  Communication  Constraints,”  IEEE  Transactions  on  Informat *.  Theory, 
Vol.36,  pp.  241-255,  March  1990. 

[36]  C.CLee  and  J.J.Chao,  "Optimam  Local  Decision  Space  Partitioning  for  Distributed 
Detection,"  IEEE  Transactions  on  Aerospace  end.  Electronic  Systems,  Vol.  25,  pp.  536- 
544,  July  1989. 

[37]  A.T.Toussaint,  “  Comments  on  the  Divergence  and  Bhattacharyya  Distance  Measures 
in  Signal  Selection,”  IEEE  Transactions  on  Communications,  Vol.  20,  pp.  485,  June  1972. 

[38]  TL.Grettenberg,  “  Signal  Selection  in  Communication  and  Radar  Systems,”  IEEE 
Transactions  on  Information  Theory,  Vol.  9,  pp.  265-272,  Oct.  1963. 

[39]  A.Gersho,  “  Principles  of  Quantization,”  IEEE  Transactions  on  Circuits  and  Systems, 
Vol.  25,  pp.  427-436,  July  1978. 

[40]  P.Noll  and  R.Zelinski,  “  Bounds  and  Quantizer  Performance  in  the  Low  Bit  Rate  Re¬ 
gion,”  IEEE  Transactions  on  Communications,  Vol.  26,  pp.  300-304,  Feb.  1978. 

[41]  N.Farvardin  and  J.W.Modestino,  “  Optimum  Quantizer  Performance  for  a  class  of 
NonGaussian  Memoryless  Sources,”  IEEE  Transactions  on  Information  Theory,  Vol.  30, 
pp.  485-497,  May  1984. 

[42]  S.A.Kassam,  “  Optimum  Quantization  for  Signal  Detection,”  IEEE  Transactions  on 
Communications,  Vol.  25,  pp.  479-484,  May  1977. 

[43]  B.Aazhang  and  H.V.Poor,  “  On  Optimum  and  Nearly  Optimum  Data  Quantization  for 
Signal  Detection,”  IEEE  Transactions  on  Communications,  Vol.  32,  pp.  745-751,  Jul. 

1984. 

[44]  D.Middleton,  An  Introduction  to  Statistical  Communication  Theory,  McGraw  Hill, 
1960. 

149 


[45]  T.L.Gabriele,  “  Information  Criteria  for  Threshold  Determination,  “  IEEE  Transac¬ 
tions  on  Information  Theory ,  Vol.  12  ,pp.  484-486,  Oct.  1966. 

[46]  A. B. Martinez,  “  A  Comparison  of  Bayesian  and  Information-Optimal  Detection 
Thresholds,”  Proceedings  of  the  1987  American  Control  Conference,  Minneapolis,  MN, 
pp.  1951-1956,  June  1987. 

[47]  LY.Hoballah  and  P.K.Varshney,  “  An  Information  Theoretic  Approach  to  the  Distrib¬ 
uted  Detection  Problem,”  IEEE  Transactions  on  Information  Theory ,  Vol.  35,  pp.  988-994, 
Sep.  1989. 

[48]  D.Kazakos,V.Vannicola,and  M.Wicks,  "On  Optimal  Quantization  for  Detection," 
Proceedings  of  the  1989  Conference  on  Information  Science  and  Systems,  March  1989 

[49]  J.Max,  “  Quantizing  for  Minimum  Distortion,”  IRE  Transactions  on  Information  The¬ 
ory,  Vol.  6,  pp.  7-12,  March  1960. 

[50]  B.Picinbono  and  PDuvaut,  “  Optimum  Quantization  for  Detection,”  IEEE  Transac¬ 
tions  on  Communications,  Vol.  36,  pp.  1254-1258,  Nov.  1988. 

[51]  DJE.Boekee  and  J.GA.Van  der  Lubbe,  ’’Some  Aspects  of  Enor  Bounds  in  Feature 
Selection,”  Pattern  Recognition,  Vol.  11,  pp.  353-360,  1979. 

[52]  H.Chemoff,  ”A  Measure  of  Asymptotic  Efficiency  for  Tests  of  a  Hypothesis  Based 
on  a  Sum  of  Observations,”  AnnMath.  Stat.,  Vol.  23,  pp.  493-507,  1952. 

[53]  M.E.Hellman,  ”Probability  of  Error,  Equivocation,  and  Chemoff  Bound,”  IEEE 
Transactions  on  Information  Theory,  Vol.  16,  pp.  368-372,  July  1970. 

[54]  T.Ito,  ’’Approximate  Error  Bounds  in  Pattern  Recognition,”  Machine  Intelligence, 
Vol.  7,  pp.  369-376,  1972. 

[55]  C.H.Chen,  ’Theoretical  Comparison  of  a  Class  of  Feature  Selection  Criteria  in  Pat¬ 
tern  Recognition,”  IEEE  Transactions  on  Computers,  Vol.  C-20,  pp.  1054-1056,  Sep. 
1971. 

[56]  P.A.Devijver,  ”On  a  New  Class  of  Bounds  on  Bayes  Risk  in  Multihypothesis  Pattern 
Recognition,”  IEEE  Transactions  on  Computers,  Vol.  C-23,  pp.  70-80,  Jan.  1974. 


150 


[57]  D.Kazakos,  “  New  Error  Bounds  and  Optimum  Quantization  for  Multisensor  Distrib¬ 
uted  Signal  Detection,”  Preprint,  Electrical  Engineering  Department,  University  of  Virgin¬ 
ia. 

[58]  J.Tsitsiklis,  “  Decentralized  Detection  by  a  Large  Number  of  Sensors,”  Mathematics 
of  Control,  Signals,  and  Systems,  1,  pp.  167-182,  1988. 

[59]  CH.Chen,  ”On  Information  and  Distance  Meaures,  Error  Bounds,  and  Feature  Selec- 
non, "Information  Science,  Vol.  10,  pp.  159-171,  1976. 

[60]  D.E.Boekee  and  J.C.Ruitenbeek,  ”A  Class  Of  Lower  Bounds  on  The  Bayesian  Prob¬ 
ability  of  Error,”  Information  Science,  pp.  32-35,  1981. 

[61]  G.T.Toussaint,  ”On  the  Divergence  Between  Two  Distributions  and  the  Probability  of 
Misclassification  of  Several  Decision  Rules,”  Proceedings  of  the  Second  International 
Joint  Conference  on  Pattern  Recognition,  Copenhagen,  Denmark,  Aug.  1974. 

[62]  R.O.Duda  and  P.E.Hart,  Pattern  Classification  and  Scene  Analysis ,  John  Wiley  & 
Sons,  Inc.,  1973. 

[63]  P.R.Krishnaiah  and  L.N.Kanal,  Handbook  of  Statistics  2,  Classification  Pattern  Rec¬ 
ognition  and  Reduction  of  Dimensionality,  North-Holland,  1982. 

[64]  R.Alshwede  and  I.Csiszar,  “  Hypothesis  Testing  with  Communication  Constraints,” 
IEEE  Transactions  on  Information  Theory,  Vol.  32,  pp.  533-542,  Jul.  1986. 

[65]  T.S.Han,  “  Hypothesis  Testing  with  Multiterminal  Data  Compression,”  IEEE  Trans¬ 
actions  on  Information  Theory,  Vol.  33,  pp.  759-772,  Nov.  1987. 

[66]  T.J.Flynn  and  R.M.Gray,  “  Encoding  of  Correlated  Observations,”  IEEE  Transactions 
on  Information  Theory,  Vol.  33,  pp.  773-787,  Nov.  1987. 

[67]  R.Blahut,  Principles  and  Practice  of  Information  Theory,  Addison-Wesley,  1987. 

[68]  C.C.Lee  and  J.B.Thomas,  “  Detectors  for  Multinomial  Input,”  IEEE  Transactions  on 
Aerospace  and  Electronic  Systems,  Vol.  19,  pp.  288-296,  March  1983. 

[69]  G.R.Walsh,  Methods  of  Optimization,  John  Wiley  and  Sons,  1975. 


151 


Appendix  A 


In  this  appendix,  we  derive  the  algorithms  based  on  which  the  fusion  center  com¬ 
bines  the  decisions  received  from  the  various  local  detectors  in  order  to  make  the  global 
decision.  We  consider  both  the  hard  decision  and  the  soft  decision  cases,  i.e.,  the  cases 
with  M  =  2  and  M  >  2. 

First  Case  :  M  =  2 

Let  PDi  and  Ppj  be  the  probabilities  of  detection  and  false  alarm  at  the  local  detector 
LDj.  The  local  decision  Zj  has  the  following  conditional  distributions 


(A.l) 

m>PDfiu-PDi>u,i 

(A.2) 

The  global  decision  u0  is  made  at  the  fusion  center  based  on  the  decision  vector  U=[zj  z2 
...  z„]  by  performing  the  likelihood  ratio  test 

P  (£/]//,)  > 

~  P(U\H0)  <  11  (A3) 

Ho 

Taking  the  logarithm  of  both  sides  of  (A.3)  we  get 

Hi 

> 

log  A  (U)  <  logrt  (A.4) 

Ho 

Using  (A.l),  (A.2),  and  making  use  of  the  independence  assumption  Vtween  the  local  de¬ 
cisions  Zj,  we  get 
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A(U)  = 


Upo^-pDl) 

i=  1 _ 

i  =  1 


1  —  Z; 


1  -Z; 


Equation  (A.5)  can  also  be  written  as 


Taking  the  logarithm  of  both  sides  of  (A.6)  we  get 


log  A  (U)  =  £log 
»  =  l 


Y  1  ~  PDi 
L  \PFi  )  l 1  "  PFi 


i  -  tn 


(A.5) 


(A.6) 


(A.7) 


Expanding  the  logarithm  of  the  product  term  on  the  right  hand  side  of  (A.7)  and  simplify¬ 
ing  we  get 


logA(tf)  = 


n 


i 


+ 


(  1  -  Z,)  log 


n-pDi) 

u-'vJ 


Combining  terms  in  (A.8)  we  obtain 


log  A  (U)  =  £log 
i  =  1 


('-PDi 

{'-PFi 


+  X  z«lQg 

i=  1 


pdS'-pf) 

-rDi) 


(A.8) 


(A.9) 


Substituting  (A.9)  into  (A.4)  and  rearranging  terms  we  get 

Hi 


X  zilQg 


rDi('-pFi)  > 


Fi> 


i=  1 


PFi^~PDi)  < 

Ho 


lOgTJ  +X  lQg 
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n-pF^ 

1  ~PDiJ 


(A. 10) 


The  equivalent  test  in  (A.  10)  suggests  that  each  received  local  decision  Zj ;  Zj  =0,  1  is 
weighted  by  a  factor  that  depends  upon  the  probabilities  of  detection  and  false  alarm  cor- 
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responding  to  that  detector.  The  sum  is  then  compared  to  a  threshold. 


Under  the  identical  sensor  assumption,  i.e.,  when  Ppi  =  Pd  ?Fi  =  Pf  i  =L  2,..., 
n,  the  left  hand  side  and  the  right  hand  side  of  (A.  10)  become 


LHS  =  log 


^0 -Pp)  y 

PF^~PD)^ 


(A.  11) 


RHS  =  logrj  +  nlog 


.  J'-PF>? 
=  logTJ 


1  -P 


1  -P 


(A.12) 


Dj 


Therefore,  equation  (A.  10)  can  be  expressed  as 


X*. 

i  =  l 


Hi 

> 
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Ho 


lOgTJ 
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1  ~Pr 


PD^~Pf) 


log 


Pp(l-PD) 


(A. 13) 


Let  K  be  the  random  variable  that  represents  the  number  of  sensors  that  decide  in  favor  of 
Hj.  It  is  clear  that  K  is  nothing  but  the  sura  on  the  left  hand  side  of  (A.  13).  This  sum  takes 
on  the  values  0,  1, ...,  n.  A  k  out  of  n  fusion  rule  is  defined  as  follows  :  Decide  Hj  if  the 
number  of  sensors  that  decide  in  favor  of  Hi  is  at  least  k,  i.e.,  decide  Hj  if 

n 

^Zi>k,k  =  1,2,  ...,n  (A.  14) 

i  =  1 


It  is  clear  from  (A.  13)  and  (A.  14)  that  the  fusion  rule  at  the  fusion  center  is  a  k  out  of  n  fu¬ 
sion  rule.The  case  with  k  =  0  has  been  excluded  because  it  corresponds  to  the  trivial  case 
of  always  deciding  Hj.  Moreover,  the  case  k  =  0  violates  the  validity  of  (A.  13)  since  the 
right  hand  side  of  (A.  13)  is  a  positive  quantity  and  cannot  be  0  or  negative. 
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Second  Case  :  M  >  2 


For  the  case  when  the  number  of  quantization  levels  M  >  2,  we  restrict  our  study  to 
the  case  of  identical  detectors.  Here  we  make  use  of  the  multinomial  distributions  derived 
in  Chapter  4.  Using  (4.37)  and  (4.38),  we  see  that 


A  (U)  = 


pX  1  DX  2  pX» 

'll'  21  ‘“‘Ml 

pX\  pX 2  p^M 

'10'  20""' A/0 


(A. 15) 


Equation  (A.  15)  can  also  be  written  as 


A  (U) 


(A.  16) 


Taking  the  logarithm  of  both  sides  of  (A.  16)  we  get 


li 


log  A  (U)  =  log  J  +  X2log 


^21  ^ 
\P20  ) 


1  (Pm 
+  -+%log  p 


V'  MOJ 


(A.17) 


Substituting  (A.17)  into  (A.4)  we  obtain  the  test 


fPn\  (pi\\  (pm\\  > 

x\l°z  p-  +  *2log  p-  +  -"+*A/log  p—  <  tog7!  (A. 18) 
VioJ  V20J  VMOj  ^ 


H, 

> 

< 

H 


o 


The  equivalent  test  in  (A.  18)  indicates  that  the  fusion  center  makes  the  final  decision  by 
counting  the  number  of  sensors  x^  that  decide  in  favor  of  symbol  a^  ;  k=T,  2,...,M,  weight 
Xfc  by  a  factor  that  depends  upon  the  probabilities  of  occurrence  of  symbol  a^  under  hy¬ 
potheses  Ho  and  Hj,  sum  over  all  values  of  x^  and  compare  the  sum  to  a  threshold. 
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Appendix  B 


In  this  appendix  we  provide  a  derivation  of  equation  (4.18).  The  possible  fusion 
rules  for  a  three-sensor  system  are  the  AND  fusion  rule,  the  OR  fusion  rule  and  the  MA¬ 
JORITY  lo~ic  fusion  rule.  The  cost  corresponding  to  each  one  of  these  fusion  rules  is  giv¬ 
en  by  (4.14),  (4.15)  and  (4.16)  respectively.  Since  the  fusion  center  is  a  minimum  average 
cost  receiver,  it  implements  the  ‘best’  one  of  the  fusion  rules,  i.e.,  the  fusion  rule  with  the 
smallest  cost.  The  cost  corresponding  to  this  best  fusion  rule  is  given  by 

R3  =  mil  ( min  '  RAND,  Ror)  ,  RMAJ)  (B.  1) 


The  minimum  of  two  quantities  a  and  b  is  given  by 


min  (a,  b)  =  ^(a  + b)  -  ^\a  -  b\ 


(B.2) 


Making  use  of  (B.2),  (B.l)  becomes 


#3  -  2  (mJn  ^A/vz)>  +  rmaj)  2\m*n^AND'R°R^  Rmaj\  (B.3) 


Again  applying  (B.2)  to  (B.3)  we  obtain 


R3  ~  2  [  2  (Rand  +  ror)  ~  9  l^/tyvo  ror\+r 


MAJ 


2  (rand  +  ror)  ~  2 1 rand  ~  ror\  ~  rmaj 


(B.4) 
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Simplifying  (B.4)  we  get 


^3  ~  4  [rand  +  ror  +  2rmaj]  4 1 rand  ror\ 

~4  \RAND  +  ROR  -  ^rmaj  ~  I rand  ~  ror\  \  (B.5) 

For  simplicity  of  presentation  we  use  the  following  notation 

Q\  =  raND  +  ror  +  2rmaj  (B.6) 

Qi  ~  rand~ror  (B.7) 

@3  =  RAND+ROR~2RMAJ  (B.8) 

Substituting  (4.14),  (4.15),  and  (4.16)  into  (B.6)  we  get 

Qi  =  C00kq[4  +  2P3f-3Pf-3P2f]  +  Cl0nQ[-2P3F+3PF+3P2F) 

+  Co,  * 1 1 W*  +  3/4  -  2P3m]  +  Cj !  ic  1  [ 4  +  2P I  -  3Pm  -  3P 2M] 

(B.9) 

Adding  and  subtracting  Pp  to  the  first  two  terms  on  the  right  hand  side  of  (B.9)  and  Pm  to 
the  last  two  terms  we  get 
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Q\  =  c00*0  [4-4Pf  +  2P3f-3P2f  +  Pf]  +  C10kq[4Pf- Pf-2P3+3P2f] 

+  C0lICl  [4  PM  -PM-  2  Pi  +  +  C117t1  [4  -  4PM  +  2Pl  -  3P2M  +  PM) 

(B.10) 

Dividing  (B.10)  by  4  and  expanding  terms  we  get 

Q\ 

“4"  =  Coo7C0(1-Pf)  +  C107C0PF+ CojTIjP^  +  Cn7tj  (1  P M) 


Coo*0 


[2P3F-3P2+PF]  - 


Cio*o 


[PF~3P2F  +  2P3F] 


~~4~ 


[r*-3r2u  +  2Fl,) 


+ 


Cn*  1 
4 


(B.ll) 


It  is  to  be  observed  that  the  first  term  on  the  right  hand  side  of  (B.  1 1 )  is  the  cost  corre¬ 
sponding  to  a  single  sensor  system.  This  cost  was  defined  in  (4.12).  Rearranging  terms  in 
(B.ll)  we  get 


Qx 


Itr 


K1--?PfO-3Pf  +  2P2f)  (C10-C oo) 


7t 


-x'>«0-3P„  +  2/>«)<Co1-C1,) 


(B.12) 


Now  we  consider  (B.7).  Substituting  (4.14),  (4.15),  and  (4.16)  into  (B.7)  we  get 
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Q2  =  C00n0[3PF-3pj:]  +Cm^0[iP2F-3PF] 

+  C01*1(3^-3/>*)+C11*1[3J>1,-3/>J,]  (B.13) 


Rearranging  terms  in  (B.13)  we  get 


Qi  =  3W1  -PM)  (C01-Cn)  -3W(1-/V)  (C^-Coo) 


(B.14) 


Dividing  both  sides  of  (B.14)  and  taking  the  absolute  value  we  obtain 

^  (Cc-Cnj-noPfd-^)  <C.o-Coo)| 

(B.15) 


Substituting  (4.14),  (4.15),  and  (4.16)  into  (B.8)  we  get 

Q3  =  C00h0[-6P3f  +  9P2f-3Pf]  +C10k0[6P3f+3Pf-9P2f] 


+  C0i7tj  [6PW+ 3PM- 9P^]  +Cn7tj  [~6/>M  +  9/,Af  3/*w]  (B.16) 

Rearranging  terms  in  (B.16)  and  dividing  by  4  we  get 
0-2  3  7Un  37C1 

T  =  ■x',f(l-3Pf+  2P2f)  (C01-Cn) 

(B.  17) 


Substituting  (B.12),  (B.15),  and  (B.17)  into  (B.5)  we  readily  obtain  (4.18). 


159 


Appendix  C 


In  this  appendix  we  derive  the  relationships  between  the  regional  threshold  and 
the  local  thresholds  T2i_]and  t2i  for  the  regional  decision  maker  RDj  in  the  hierarchical 
system  S4.  This  relationship  is  important  in  carrying  out  the  design  of  the  optimum  sys¬ 
tem.  We  start  with  (5.32)  which  is  repeated  here  for  convenience 


~  A2i-lz2i-l+A2iz2i 


(C.1) 


Recall  that  the  random  variables  z^  and  z2i  take  on  the  values  0  and  1.  The  random  vari¬ 
able  Lj  takes  on  the  value  0  only  when  z^i-i  =  0  and  z2j  =  0.  Since  z2j.  1  and  z2l  are  inde¬ 
pendent,  we  may  write 

P  (L,-  =  0|  H0 )  =  P  (z2|._  ,  =  0|  H0)  P  (z2i  =  0|  H0)  (C.2) 


P  (Li  =  0|  //j)  =/>(z2i_1  =  0|//1)P(z2,.  =  0|//1)  (C.3) 

By  making  use  of  (5.2)  and  (5.3),  the  above  two  probabilities  become 

P  (Lt  =  0|  H0)  =  ( 1  -  PFL2i  _,)(!-  PFL2i )  (C.4) 

P{Li  =  0\Hl)  =  (1—1° DL2i _ j )  ( 1  - f DL2i)  (C.5) 

In  a  similar  manner,  we  can  obtain  the  rest  of  the  values  that  Lj  assumes  alongwith  their 
probabilities  under  Hg  and  Hj.  These  are  shown  in  Table  Cl  below. 
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P(LilHo) 


PCLjlH,) 


Li 

0 

A2i-1 

A2i 

A2i-1+A2i 


(l-PFL^.jXl-PFI^i) 

PFL^.id-PFl^i) 

(l-PFl^i.OPFLzi 

PFL^iPFL^j 


( 1  -PDL^j. ! )( 1  -PDL2i) 

PDI^j.jd-PDLai) 

( 1  -PDL^j.  i  )PDL2j 
PDL^i.jPDL^i 


Table  Cl  :  Probability  distribution  of  Lj 


The  values  that  Lj  assumes  arranged  in  an  ascending  order  are  shown  in  the  figure  below. 


ZiO  Zj!  Zj2  Zj3  Zj4 

- x - x - x - — x -  Lj 

0  min(A2i.i,  A2,)  max(A2i.1,  A2i)  A2i-1+A2i 


By  examining  (5.31),  we  see  that  both  A2i-i  and  A2j  are  independent  of  the  regional 
threshold  t||  while  the  equivalent  threshold  Q  is  a  function  of  T||.  Since  at  the  moment 
there  are  no  restrictions  on  the  values  assumed  by  Tij,  then  this  means  that  Q  also  may  as¬ 
sume  any  value.  Therefore,  the  possible  values  of  Lj  partition  the  space  of  Cj  into  five  re¬ 
gions.  If  the  threshold  Q  lies  in  ZjQ,  then  according  to  (5.33),  PFRj  =  PDRj  =  1  (here  all  the 
values  of  Lj  are  greater  than  Cj  and,  therefore,  P  {Lz-  >  Q  }  =  1).  This  also  means  that  1- 
PFRj  =  1-PDRj  =  0.  Let  us  assume  that  regional  detector  RDi  is  the  only  detector  with 
PFRj  =  PDRj  =  1.  All  other  regional  detectors  may  have  other  values  of  the  false  alarm  and 
detection  probabilities.  Substituting  these  probabilities  into  (5.4)  results  in  the  value  of  the 
product  term  to  be  equal  to  0.  This  means  that  the  value  of  the  second  term  on  the  right 
hand  side  of  equation  (5.5)  is  0.  Therefore,  under  these  conditions  the  value  of  the  MAC 
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,R,  is  equal  to  Rq  which  is  the  maximum  possible  value  of  R.  This  undesirable  situation 
occurs  when  Cj  <  0.  Using  the  definition  of  Q  in  (5.31),  we  obtain 


,  _  ( 1  -  PFL2i  _ , )  ( 1  -  PFL2i)  ^ 
°gT1‘  (l -PDL2i_x)  (1  -PDL2i)  -° 


(C.6) 


Expanding  the  logarithm  of  the  product  of  two  terms  and  rearranging  the  terms  in  (C.6), 
we  can  express  the  condition  Cj  <  0  in  terms  of  T|j  as 

(1  ~ PDL^i^x)  (1  -PDL2i)  n 
T*i  ^  ( 1  -  PFL2i _  j )  ( 1  -  PFL2i)  ~  il  (C7; 


Similarly,  if  Cj  is  such  that  Cj  e  Zj4,  then  PFR,  =  PDRj  =  0  (in  this  case  all  the  values  of  Ll 
are  smaller  than  Ct-  and,  therefore,  P  {L,  <  C, )  =  0.  Using  these  probabilities  in  evaluating 

(5.4) ,  we  see  that  the  product  term  is  0  and  that  the  second  term  on  the  right  hand  side  of 

(5.5)  is  also  0.  This  situation  will  lead  to  the  result  that  the  global  cost  R  =  Rq  which,  of 
course,  is  not  a  desirable  result.  The  values  of  Q  that  lead  to  this  undesirable  situation  are 
given  by  Q  >  A2j_i  +A2j.  This  condition  can  be  expressed  as 


(\-PFL2i_l)(l-PFL2i  a 
'(\-PDL2i_x)  (\-PDL2i)  A2'-1  *2, 


(C.8) 


From  the  definitions  of  A2,.j  and  A2,  in  (5.31),  we  can  simplify  (CH'i  so  that  we  get  the 
following  undesired  values  of  T}; 


^  PDL2i  _  j  PDL2i 


(C.9) 


Since  ■Hi  ^Dil  and  rjj  >  result  in  the  maximum  possible  value  of  the  MAC,  t|j  should 
not  assume  values  in  these  regions.  If  Tij  corresponds  to  the  values  of  Cj  such  that 
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0  <  Cj  <  min  (^.i , 


(C.10) 


then  the  system  MAC  remains  the  same  for  all  possible  values  of  in  this  region.  This  is 
due  to  the  fact  that  PFRj  and  PDR,  remain  the  same  in  this  region.  Substituting  into  (C.  10) 
the  value  of  Cj  given  in  (5.31)  we  get 


0  <  logTj.  +  log 


(1  -PFL2i_x)  ( 1  -  PFL2i) 
(1  -PDL2i_x)  ( 1  - PDL2i) 


<min  (A2i.vA2i) 


(C.11) 


By  using  the  definition  of  Djj  in  (C.7),  we  can  express  (C.  11)  as 

log Dn  <  logT|(-  <  logDn  +  min  (A2i_  v  A2i )  (C.12) 


Therefore,  In  terms  of  rjj,  the  region  defined  by  (C.10)  can  be  expressed  as 

Dn<T]i<Di2  (C.13) 

where 

Di2  ~  Dn^p(min(A2i_lyA2i)) 

The  two  remaining  regions  of  i\[  are  obtained  from  the  following  regions  of  C, 

min(A2i_vA2i)  <Ci<max(A2i_x,Ali)  (C.14) 

max  (A2i_  v  A2i)  <Ci<A2i_l+A2i  (C.15) 

Similar  to  what  we  did  above,  these  two  regions  can  be  expressed  in  terms  of  r|j  as 

Di2<r\i<Di\exp[max(A2i_vA2i)]  =  Di3  (C.16) 


D,3<t1;<D,.4 


(C.17) 


The  above  partitions  for  t|j  can  be  used  in  the  optimization  process. 
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Appendix  D 

In  this  appendix  we  prove  the  inequalities  given  by  (6.32)  and  (6.50).  We  begin  with 
the  inequality  given  (6.32).  This  inequality  can  be  verified  as  follows.  Assume  for  the  mo¬ 
ment  that  (6.32)  is  true.  This  inequality  can  be  written  as 

A  PD  A  PF 

(\-PD)  >  (l-PF)  W 


Integrating  each  side  of  (D.l)  with  respect  to  the  variable  involved  we  get 

P*  J  n  Pf  Jn 

r  dPD  ,  dPF 

J  (1-PD)  >  I  (\-PF)  W 

Performing  the  integration  in  (D.2)  we  get 

-In  (l  -PD)  >-tn(l  -PF)  (D.3) 


This  inequality  can  be  written  as 

ln(l-PF)>ln(l-PD) 


(D.4) 


Since  the  logarithmic  function  In  (t)  is  an  increasing  function  of  t,  then  from  (D.4)  we 
readily  obtain  the  following  inequality 


P D  >  P  F 


(D.5) 


The  inequality  in  (D.5)  is  known  to  be  true  because  of  the  concave  nature  of  the  receiver 
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operating  characteristic  of  the  local  detectors  indicating  that  the  inequality  given  by  (6.32) 
is  true  indeed 

Next,  we  prove  the  inequality  given  by  (6.50).  This  inequality  can  be  verified  as  fol¬ 
lows.  First  assume  that  (6.50)  is  true  and  rewrite  it  in  the  equivalent  form 


(D.6) 


The  right  hand  side  of  (D.6)  represents  the  slope  of  the  receiver  operating  characteristic  of 
the  local  detector,  while  the  left  hand  side  represents  the  ratio  of  the  detection  to  the  false 
alarm  probabilities  at  the  point  where  we  evaluate  the  slope.  Since  the  receiver  operating 
characteristic  is  a  concave  function,  it  follows  that  the  curve  representing  the  slope  lies 
above  the  curve  representing  the  ratio  of  the  detection  to  the  false  alarm  probabilities  for 
all  points.  This  is  clear  from  Fig.  6.2.  It  follows  immediately  that  (D.6)  is  true,  and  there¬ 
fore  our  initial  assumption  regarding  (6.50)  is  also  true. 
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MISSION 

OF 

ROME  LABORATORY 

Rome  Laboratory  plans  and  executes  an  interdisciplinary  program  in  re¬ 
search,  development,  test,  and  technology  transition  in  support  of  Air 
Force  Command,  Control,  Communications  and  Intelligence  (C  I)  activities 
for  all  Air  Force  platforms.  It  also  executes  selected  acquisition  programs 
in  several  areas  of  expertise.  Technical  and  engineering  support  within 
areas  of  competence  is  provided  to  ESD  Program  Offices  (POs)  and  other 
ESD  elements  to  perform  effective  acquisition  of  C  I  systems.  In  addition, 
Rome  Laboratory's  technology  supports  other  AFSC  Product  Divisions,  the 
Air  Force  user  community,  and  other  DOD  and  non-DOD  agencies.  Rome 
Laboratory  maintains  technical  competence  and  research  programs  in  areas 
including,  but  not  limited  to,  communications,  command  and  control,  battle 
management,  intelligence  information  processing,  computational  sciences 
and  software  producibility,  wide  area  surveillance/sensors,  signal  proces¬ 
sing,  solid  state  sciences,  photonics,  electromagnetic  technology,  super¬ 
conductivity,  and  electronic  reliability/maintainability  and  testability. 


